-- Logs begin at Wed 2022-11-02 11:43:19 UTC, end at Wed 2022-11-02 12:48:29 UTC. -- Nov 02 12:13:34.333648 dv-cinder-863273-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 12:13:34.346165 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 12:13:34 2022] *** Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: nodename: dv-cinder-863273-1-iscsi Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: machine: x86_64 Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: clock source: unix Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: pcre jit disabled Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: detected number of CPU cores: 4 Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: current working directory: / Nov 02 12:13:34.348591 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: detected binary path: /usr/bin/uwsgi-core Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: your processes number limit is 47680 Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: your memory page size is 4096 bytes Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: detected max file descriptor number: 2048 Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: lock engine: pthread robust mutexes Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: thunder lock: enabled Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 12:13:34.349590 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 12:13:34.394077 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: Python main interpreter initialized at 0x5592838449c0 Nov 02 12:13:34.394077 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: python threads support enabled Nov 02 12:13:34.394077 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: your server socket listen backlog is limited to 100 connections Nov 02 12:13:34.394077 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: your mercy for graceful operations on workers is 90 seconds Nov 02 12:13:34.394638 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 12:13:34.394638 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: *** Operational MODE: preforking *** Nov 02 12:13:34.394872 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: *** uWSGI is running in multiple interpreter mode *** Nov 02 12:13:34.394872 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: spawned uWSGI master process (pid: 81471) Nov 02 12:13:34.395139 dv-cinder-863273-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 12:13:34.396431 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 1 (pid: 81480, cores: 1) Nov 02 12:13:34.396431 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 2 (pid: 81481, cores: 1) Nov 02 12:13:34.396431 dv-cinder-863273-1-iscsi devstack@keystone.service[81471]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 12:13:38.823134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 12:13:38.823134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Nov 02 12:13:38.824304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 12:13:38.824304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: from cryptography.utils import int_from_bytes Nov 02 12:13:38.824304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 12:13:38.824304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: from cryptography.utils import int_from_bytes Nov 02 12:13:38.825354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 12:13:38.825354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Nov 02 12:13:38.876602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 12:13:38.876866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 12:13:38.876866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 12:13:38.877190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 12:13:38.877334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] command line args: [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 12:13:38.877467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] command line args: [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 12:13:38.877598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 12:13:38.877718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 12:13:38.877718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ================================================================================ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 12:13:38.877973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ================================================================================ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 12:13:38.878111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] admin_token = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.878234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.878365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_dir = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.878496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_dir = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.878638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.879076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.879274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_source = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.879444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_source = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.879619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] debug = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.879774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] debug = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.879945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.880305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.880305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.880645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.880829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.881039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.881219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.881382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.881568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.881724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.881906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_config_append = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_config_append = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_dir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.882957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_dir = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.883079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.883205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.883334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.883458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.883596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.884417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.884541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.884726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.884851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.885065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.885210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.885420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.885420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.885823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.885950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.886145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.886145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.886449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.886578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.886757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.886893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.887085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.887262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.887466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.887655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.887834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.887965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.888947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] publish_errors = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] publish_errors = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.889910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] standard_threads = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.890910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] standard_threads = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_journal = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.891929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_journal = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_json = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_json = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_stderr = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_stderr = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_syslog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_syslog = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.892898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 12:13:38.893053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.893980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.894967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.895971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.896973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.897153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.897399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.897532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.897684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.897809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.897937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.898977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.899988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.900117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.900238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.900409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.900531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.900726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.900862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.901958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.902988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.903111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.903247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.903381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.903507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.903747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.903876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.904892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.905075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.905259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.905434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.905600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.905765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.905927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.906089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.906253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.906432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.906609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.906787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.906952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.907120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.907281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.907472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.907636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.907799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.907960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.908131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.908362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.908547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.908711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.908886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.909097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.909274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.909454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.909625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.909782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.909939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.910114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.910287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.910461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.910630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.910817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.911005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.911187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.911354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.911520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.911692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.911865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.912031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.912194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.912364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.912527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.912688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.912865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.913158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.913342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.913503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.913668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.913852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.914047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.914213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.914376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.914532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.914715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.914884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.915061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.915218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.915399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.915573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.915573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.915882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.916065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.916239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.916402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.916402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.916731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.916908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.917106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.917261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.917261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.917556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.917740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.917740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.918089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.918283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.918465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.918646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.918829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.919015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.919187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.919362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.919537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.919714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.919890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.920078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.920370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.920573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.920755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.920933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.921141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.921321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.921518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.921706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.921706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.922060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.922236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.922413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.922608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.922794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.923139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.923326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.923509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.923684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.923864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.924057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.924242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.924424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.924604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.924791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.924973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.925262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.925447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.925447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.925808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.925984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.external = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.926158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.926357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.926539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.926767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.927003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.927190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.927367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.927539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.password = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.927707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.927937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.token = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.928107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.928280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.928280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.928603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.external = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.928785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.929024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.929187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.929363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.929557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.929739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.929969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.930165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.password = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.930343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.931005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.token = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.931189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.931189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.931547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.931739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.931924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.932915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.933076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.933076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.933388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.933517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.933517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.933517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.934933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.935890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.936936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.937940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.938954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.939185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.939312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.939440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.939691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.939691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.939925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.940953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.941293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.941423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.941550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.941689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.941689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.941925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.942893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.943901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.944975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.945965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.946084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.946798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.946911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.947924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.948973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.949953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.950940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.951936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.952986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.953973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.954949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.955964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.956942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.957921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.958996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.959973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.960965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.961935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.962860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.963868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.964952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.965923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.966907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.967970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.968936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.969984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.970969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.971915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.972955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.973975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.974985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.975882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.976932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 12:13:38.977932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 12:13:38.998861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:38.999116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:38.999306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.000979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.001302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.001440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.008442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.010679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.042237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.042528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.045982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.046218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.099884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.100769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.151683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.151926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.151926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.152154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.313397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.313673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.313837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.313982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.314955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.315090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.340658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.340912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.341936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.342067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.342203 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.342349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.342480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.342480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.354539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.354775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.368716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.368971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.369217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.369384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.369384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.369668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.369806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.369940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.370073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.370205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.370337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.370483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.370621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.370753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 12:13:39.388441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.388699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.388861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 12:13:39.389023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 12:13:39.389155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.389289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.389423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 12:13:39.389555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 12:13:39.389708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.389841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.389973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.390105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.390238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.390390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.390522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.390652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.400501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.400709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.400845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.401055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.401197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.401332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.401459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.401591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.408383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.408583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.411321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.411556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.411701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.411864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.412923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.413895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.414025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.414142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.414340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.437184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.437601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.438682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.438974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.439115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.439401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.443680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.443969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.444133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.444271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.444410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.444677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.451198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.451493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.451638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.451907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.457576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.457886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.458038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.458217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.458354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.458483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.458611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.458741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.458892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.459866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.475218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.475502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.475688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.475841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.479462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.479875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.483129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.483441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.483659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.483866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.487751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.488059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.490934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.491332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.494757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.495064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.495275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.495504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.495712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.495916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.496116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.496321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.496534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.496732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.496947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.496947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.512559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.512913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.513143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.513362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.513556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.513737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.513925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.514233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.520576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.521307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.522183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.522895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.523710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.523987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.524463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.524743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.526780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.527070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.527363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.527567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.527708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.527942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.528162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.528458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.528684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.528684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.539914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.540276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.540474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.540673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.540900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.541137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.541351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.541551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.541757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.541960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.542157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.542356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.542356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.542746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.557757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.558436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.560708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.561462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.563649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.564490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.565219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.565985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.566696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.566980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.567390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.567618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.567822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.568135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.574575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.575254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.576068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.576776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.579032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.579446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.579739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.579933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.580102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.580271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.580436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.580698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.587938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.588244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.588532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.588668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.588972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.589294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.595511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.595801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.596088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.596253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 12:13:39.596392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.596522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.596653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.596787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.596916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.597978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598504 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.598935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.599072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.599192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.599318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.599445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.599795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 12:13:39.634933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5592838449c0 pid: 81481 (default app) Nov 02 12:13:39.635568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5592838449c0 pid: 81480 (default app) Nov 02 12:13:39.638424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573c69-8310-46a6-8211-67db03dc5f56 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:39.638706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573c69-8310-46a6-8211-67db03dc5f56 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:39.638979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573c69-8310-46a6-8211-67db03dc5f56 None None] PATH_INFO: `/v3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:39.643170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1/1] 10.222.0.85 () {50 vars in 791 bytes} [Wed Nov 2 12:13:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:13:46.806450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:46.807002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:46.807002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:46.811158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1/2] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:13:46.817383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:46.818383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:46.818383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:46.892651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: INFO dbcounter [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] Registered counter for database keystone Nov 02 12:13:46.913390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG oslo_db.sqlalchemy.engines [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81481) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 02 12:13:46.920615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writer thread running {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 12:13:47.107823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG passlib.handlers.bcrypt [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81481) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 12:13:47.108125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG passlib.handlers.bcrypt [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81481) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 12:13:47.169113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:47.170737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:47.313328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 2/3] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 12:13:47.839987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:47.910511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: INFO dbcounter [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] Registered counter for database keystone Nov 02 12:13:47.934039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG oslo_db.sqlalchemy.engines [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81480) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 02 12:13:47.941024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writer thread running {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 12:13:48.063945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07737eb0-ed17-415e-9a28-c43228511087 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:48.068601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:48.069284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:48.069565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:48.069866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:48.079744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:48.132360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 12:13:48.132659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529adbf-3ef7-4823-a0cb-d54b1b9acec0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:48.143795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 2/4] 10.222.0.85 () {58 vars in 1324 bytes} [Wed Nov 2 12:13:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 12:13:48.150240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:48.168646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b533f-4fa8-4531-9797-974b0b9e5758 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:48.173109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:48.174250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:48.174529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:48.174799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] PATH_INFO: `/v3/projects/bbb8bdae345348eca7be88e01d7954f8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:48.176432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] RBAC: Authorizing `identity:get_project(project_id=bbb8bdae345348eca7be88e01d7954f8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:48.230752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING keystone.common.rbac_enforcer.enforcer [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 12:13:48.231096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffe54b-6134-4fb4-85fd-e2f39f24bec0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:48.235773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 3/5] 10.222.0.85 () {56 vars in 1207 bytes} [Wed Nov 2 12:13:48 2022] GET /identity/v3/projects/bbb8bdae345348eca7be88e01d7954f8 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:13:50.485073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d06ad-d8a6-4a92-a828-984de8eeb574 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:50.485663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d06ad-d8a6-4a92-a828-984de8eeb574 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:50.486032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d06ad-d8a6-4a92-a828-984de8eeb574 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:50.494155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 3/6] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:13:50.500252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:50.500788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:50.500918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:50.523805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:50.525570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:50.533902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 4/7] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 12:13:50.807017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d06ad-d8a6-4a92-a828-984de8eeb574 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:50.820635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d06ad-d8a6-4a92-a828-984de8eeb574 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:50.825192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e881c9-2844-4440-a1c8-cd72a68b11d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:50.825820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e881c9-2844-4440-a1c8-cd72a68b11d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:50.826097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e881c9-2844-4440-a1c8-cd72a68b11d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:50.826372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e881c9-2844-4440-a1c8-cd72a68b11d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:50.835392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e881c9-2844-4440-a1c8-cd72a68b11d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:50.837351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e881c9-2844-4440-a1c8-cd72a68b11d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:50.847427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 4/8] 10.222.0.85 () {58 vars in 1324 bytes} [Wed Nov 2 12:13:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:13:50.853669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:50.863738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e26bfc-fb7e-478e-9e96-24ef766ffd8f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:50.869290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473435f-b2cf-423e-aa08-29be9a5ab8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:50.870010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473435f-b2cf-423e-aa08-29be9a5ab8eb admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:50.870333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473435f-b2cf-423e-aa08-29be9a5ab8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:50.870652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473435f-b2cf-423e-aa08-29be9a5ab8eb admin admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:50.872183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473435f-b2cf-423e-aa08-29be9a5ab8eb admin admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:50.874213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473435f-b2cf-423e-aa08-29be9a5ab8eb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:50.875997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 5/9] 10.222.0.85 () {56 vars in 1198 bytes} [Wed Nov 2 12:13:50 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:13:53.599777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2242d-0a29-4fa6-b279-75f64581f283 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:53.601153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2242d-0a29-4fa6-b279-75f64581f283 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:53.601264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2242d-0a29-4fa6-b279-75f64581f283 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:53.603719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 5/10] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:13:53.606265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e792d-efed-4dcf-b1f3-1691f77a5d4e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:53.606733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e792d-efed-4dcf-b1f3-1691f77a5d4e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:53.607209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e792d-efed-4dcf-b1f3-1691f77a5d4e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:53.609736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 6/11] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:13:53.612817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:53.613313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:53.613725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:53.616801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:53.617340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:53.617835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:53.648079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:53.650214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:53.679213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81480) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 12:13:53.679767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81480) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 12:13:53.730466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:53.732049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:53.760303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 7/12] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:13:53.848533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 6/13] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:13:54.030120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.060532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:54.124789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.157513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b58949-4e96-42d7-8ae9-6f42e72becdb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.200531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.201176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.201452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.201720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.279007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:54.280670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:54.300173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19b06-9719-44f8-ac3c-f77c06b7dbc9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.303205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:54.316980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:54.343711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.344846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.345166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.345440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.359741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:13:54.368947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:13:54.370877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:13:54.372748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 7/14] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:13:54.376330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.426984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 8/15] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:13:54.433325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.489708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25530a4b-16e6-4529-8287-1e4b589e0ae5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.526384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.527334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.527781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.528417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.529571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:13:54.530986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:54.533686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:54.549293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 8/16] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:13:54.553700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5439b-0141-45c6-9d21-695dacc6ab1f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.555430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.585265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:54.588768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:54.606310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.607318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.607762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.608124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.620497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:54.622513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:13:54.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:13:54.644811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:13:54.646441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:13:54.646441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:13:54.646441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:13:54.646441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 9/17] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:13:54.650885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.674545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cfe36-66dc-443b-9fd4-c4ee88182063 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.712879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.713584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.713868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.714145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.716333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:54.717898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:54.720550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 9/18] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:13:54.726141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.809301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499f48b-c810-42f7-bc4c-4a4473ba9b19 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.837518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:54.847955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.848576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.848857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.849157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.849814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:13:54.850055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4274e4-0f8d-4bc8-8745-3f5d8d9437c8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:54.850233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:54.851536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:54.874581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 10/19] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:13:54.910417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc62009-5397-4844-93fb-0b4867256c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:54.911382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc62009-5397-4844-93fb-0b4867256c3f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:54.911799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc62009-5397-4844-93fb-0b4867256c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:54.912215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc62009-5397-4844-93fb-0b4867256c3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:54.914768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc62009-5397-4844-93fb-0b4867256c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:54.917320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc62009-5397-4844-93fb-0b4867256c3f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:54.920004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 10/20] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:13:54.927074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:54.987303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:55.092679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c5840-75ba-4b7c-8795-a69da0c5094d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:55.139095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:55.139549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:55.139944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:55.140326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:55.141281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.domain.id=default, user.id=16d94c9123ce4b32b47b3b412b38ab1b) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:13:55.141725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:55.144306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdceb7-c831-43dc-8413-76ae876da58b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:55.154986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 11/21] 10.222.0.85 () {56 vars in 1322 bytes} [Wed Nov 2 12:13:54 2022] GET /identity/v3/role_assignments?user.id=16d94c9123ce4b32b47b3b412b38ab1b&scope.domain.id=default&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 233 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:13:57.722258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e735b-9ca9-4678-8260-3786e78c34fc None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:57.722958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e735b-9ca9-4678-8260-3786e78c34fc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:57.723198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e735b-9ca9-4678-8260-3786e78c34fc None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:57.726944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 11/22] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:13:57.737188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369046f-cbc4-4cc3-8d9c-a74f4241d5b2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:57.738705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369046f-cbc4-4cc3-8d9c-a74f4241d5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:57.739482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369046f-cbc4-4cc3-8d9c-a74f4241d5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:57.786884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7369046f-cbc4-4cc3-8d9c-a74f4241d5b2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:57.789504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7369046f-cbc4-4cc3-8d9c-a74f4241d5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:57.956507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 12/23] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:13:58.815454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a2af-5b61-4670-906d-d4cc5bb6db55 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:58.816956 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a2af-5b61-4670-906d-d4cc5bb6db55 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:58.817819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a2af-5b61-4670-906d-d4cc5bb6db55 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:58.820838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 12/24] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:13:58.830821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:58.832397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:58.833258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:58.874034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:58.875255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:58.875747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a2af-5b61-4670-906d-d4cc5bb6db55 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:58.913517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:59.058760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 13/25] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:13:59.226337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a2af-5b61-4670-906d-d4cc5bb6db55 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:59.311720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:59.312810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:59.313340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:59.313795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:59.361077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:59.364167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-39e278ae-550c-423f-b19f-61e16d9e4bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:13:59.405182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:13:59.406786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:13:59.408324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:13:59.408324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:13:59.408324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:13:59.408324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 13/26] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 534 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:13:59.418672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:59.615785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819b88-bac5-41fe-9587-bb02d9a256ed None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:13:59.671026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534842c-6c6d-4a95-bbd9-7c786f16f0f7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:59.671513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534842c-6c6d-4a95-bbd9-7c786f16f0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:59.672022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534842c-6c6d-4a95-bbd9-7c786f16f0f7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:59.679159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:13:59.679782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:59.680062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:59.680322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:59.681012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:13:59.681329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:13:59.682733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:13:59.683396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 14/27] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:13:59 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:13:59.691787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2db4c-5eeb-4429-a936-e7b83c3f1110 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:13:59.692284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2db4c-5eeb-4429-a936-e7b83c3f1110 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:13:59.692781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2db4c-5eeb-4429-a936-e7b83c3f1110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:13:59.707133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 14/28] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:13:59.715876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:13:59.762864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3dc2db4c-5eeb-4429-a936-e7b83c3f1110 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:13:59.765279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3dc2db4c-5eeb-4429-a936-e7b83c3f1110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:13:59.766319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:59.785827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:13:59.968984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 15/29] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:00.000660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131477b9-f6e3-462e-b584-79def7f4e489 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:00.009299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747693-3a51-4b4c-923e-336bed0d5615 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.009774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747693-3a51-4b4c-923e-336bed0d5615 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.010263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747693-3a51-4b4c-923e-336bed0d5615 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.012890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 16/30] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:00.027289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.027750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.028222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.066614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ab534-10a2-4d9a-8600-1cde9f23c2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:00.068069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab534-10a2-4d9a-8600-1cde9f23c2ba None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.068842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab534-10a2-4d9a-8600-1cde9f23c2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.069673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ab534-10a2-4d9a-8600-1cde9f23c2ba None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.071096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ab534-10a2-4d9a-8600-1cde9f23c2ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:00.073908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ab534-10a2-4d9a-8600-1cde9f23c2ba None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:00.091500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:00.097573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:00.344902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 17/31] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:00.346327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:00.377622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 15/32] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:13:59 2022] POST /identity/v3/projects => generated 301 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:14:00.520594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:00.540410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7717c-8819-498a-9f1f-c728e56ef7b3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.540855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7717c-8819-498a-9f1f-c728e56ef7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.541357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7717c-8819-498a-9f1f-c728e56ef7b3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.544156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 16/33] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:00.553607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4140b-fc5d-4933-9df2-6361abe6e03d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:00.555782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.556324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.556848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.610838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:00.611764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.612248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.612727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.615808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:00.621376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:00.638527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:00.647316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:00.648261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:00.649060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:00.654657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 18/34] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:00.657818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:00.785983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:00.797186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 17/35] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:00.799245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:00.897363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f520eb-1775-4b26-8a7f-7183c60f86d0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:00.967461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:00.968763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:00.969418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:00.970891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:00.972302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:00.975307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:00.982610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:00.990383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71eb8-fdd5-4d46-a3c9-bc2d683cfe42 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:01.061468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:01.063595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.064251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.064905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.066328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:01.067101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:01.070219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:01.094455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 18/36] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:01.096482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:01.193651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 19/37] 10.222.0.85 () {60 vars in 1131 bytes} [Wed Nov 2 12:14:00 2022] POST /identity/v3/roles => generated 225 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:01.194486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:01.197164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:01.288516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b330b1-3b6f-4a42-8ee8-4781ea0c6bb8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:01.366558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:01.367784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.368247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.368708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.369890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:01.372352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:01.401442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 19/38] 10.222.0.85 () {60 vars in 1131 bytes} [Wed Nov 2 12:14:01 2022] POST /identity/v3/roles => generated 219 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:01.403080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:01.423466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f6ab-34bd-4b6b-804d-5196adc0910a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:01.504936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c58a-0da0-4d7c-9bb4-d61ae56474f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:01.506668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c58a-0da0-4d7c-9bb4-d61ae56474f0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.507153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c58a-0da0-4d7c-9bb4-d61ae56474f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.507621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c58a-0da0-4d7c-9bb4-d61ae56474f0 None admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.510156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035c58a-0da0-4d7c-9bb4-d61ae56474f0 None admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:01.512615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035c58a-0da0-4d7c-9bb4-d61ae56474f0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:01.516273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 20/39] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:01 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:01.517972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8f202-ef10-4ef9-a91f-d602b7b77955 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.518266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8f202-ef10-4ef9-a91f-d602b7b77955 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.518541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8f202-ef10-4ef9-a91f-d602b7b77955 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.521208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 21/40] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:01.529248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8f202-ef10-4ef9-a91f-d602b7b77955 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:01.579658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde8de3-5191-4180-9f8d-5a78bd654730 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:01.612423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:01.614911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:01.638175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f193b5b-264b-4772-8a67-86f140f2202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:01.638175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193b5b-264b-4772-8a67-86f140f2202e None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.638175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193b5b-264b-4772-8a67-86f140f2202e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.638175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193b5b-264b-4772-8a67-86f140f2202e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.638723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193b5b-264b-4772-8a67-86f140f2202e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:01.640293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193b5b-264b-4772-8a67-86f140f2202e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:01.660265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 20/41] 10.222.0.85 () {60 vars in 1131 bytes} [Wed Nov 2 12:14:01 2022] POST /identity/v3/roles => generated 223 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:01.661925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.662447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.662726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.674817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8f202-ef10-4ef9-a91f-d602b7b77955 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:01.731882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:01.732971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.733604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.734095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.735914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:01.737931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:01.740031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 22/42] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:01.746723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:01.804139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:01.805952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:01.891127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cade985-ed91-4d0b-8703-adf386034c82 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:01.906721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 21/43] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:01.931357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:01.940696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:01.941864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:01.942337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:01.942796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] PATH_INFO: `/v3/domains/default/users/16d94c9123ce4b32b47b3b412b38ab1b/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:01.946526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=16d94c9123ce4b32b47b3b412b38ab1b, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:01.948031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:01.973339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 23/44] 10.222.0.85 () {58 vars in 1361 bytes} [Wed Nov 2 12:14:01 2022] PUT /identity/v3/domains/default/users/16d94c9123ce4b32b47b3b412b38ab1b/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:14:02.183823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:02.289847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e54e1-adcd-44c2-8709-4c2ac7ece824 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:02.304746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:02.329852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd12ea-589d-4598-a7e4-15716f7cc2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:02.330501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd12ea-589d-4598-a7e4-15716f7cc2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:02.330777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd12ea-589d-4598-a7e4-15716f7cc2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:02.331051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd12ea-589d-4598-a7e4-15716f7cc2f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:02.332222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd12ea-589d-4598-a7e4-15716f7cc2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:02.333775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd12ea-589d-4598-a7e4-15716f7cc2f7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:02.335389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 22/45] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:02.341332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:02.460637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10373866-8064-473d-a8e4-02bae396ad94 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:02.497450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6a6ee-f70c-4138-bf0a-b841388a3490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:02.498067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a6ee-f70c-4138-bf0a-b841388a3490 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:02.498342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a6ee-f70c-4138-bf0a-b841388a3490 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:02.498611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6a6ee-f70c-4138-bf0a-b841388a3490 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:02.499276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6a6ee-f70c-4138-bf0a-b841388a3490 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:02.500725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6a6ee-f70c-4138-bf0a-b841388a3490 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:02.535712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 24/46] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:14:02 2022] POST /identity/v3/projects => generated 312 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:04.170672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbf958-a4e0-4376-ae2d-0feb8369268e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:04.171249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbf958-a4e0-4376-ae2d-0feb8369268e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:04.171523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbf958-a4e0-4376-ae2d-0feb8369268e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:04.173845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 23/47] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:04.182217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:04.182970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:04.182970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:04.202841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:04.204491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:04.244678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:04.282628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 25/48] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:04.599354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbf958-a4e0-4376-ae2d-0feb8369268e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:04.718366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbf958-a4e0-4376-ae2d-0feb8369268e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:04.761108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9d822-7072-469c-b7f3-a7e11d4e081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:04.761676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9d822-7072-469c-b7f3-a7e11d4e081d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:04.761960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9d822-7072-469c-b7f3-a7e11d4e081d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:04.762248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9d822-7072-469c-b7f3-a7e11d4e081d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:04.776936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9d822-7072-469c-b7f3-a7e11d4e081d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-65d9d822-7072-469c-b7f3-a7e11d4e081d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:04.783580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:04.789672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:04.791337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 24/49] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:04.794145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:04.928856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd137-105d-4841-8157-4ab43a2ea8cd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:04.929343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd137-105d-4841-8157-4ab43a2ea8cd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:04.929806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd137-105d-4841-8157-4ab43a2ea8cd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:04.932031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 25/50] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:04.940279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:04.940740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:04.941229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:04.963646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f5b48-700a-4398-b17b-92eba02423ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:04.967714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:04.969084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:05.005459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:05.006295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:05.006726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:05.007181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:05.008246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:05.008759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:05.011323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:05.013726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:05.025678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 26/51] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:05.032497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:05.063371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 26/52] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:05.080306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:05.201326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d97df7-a6cd-4ebe-89e3-6b021497ade3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:05.251791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:05.252774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:05.253271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:05.253740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:05.256260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:05.258876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:05.261682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 27/53] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:05 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:05.268398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:05.343352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:05.448175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c16131-7eee-4f25-928e-8dccbd0a0483 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:05.502793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:05.503747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:05.504174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:05.504614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:05.506991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:05.509199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:05.511114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 27/54] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:05.519827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:05.527270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e609f-f32d-4821-b54b-44d992eeae49 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:05.530765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:05.585887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:05.586623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:05.586965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:05.587301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:05.588672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:05.590300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:05.591495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:05.592526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 28/55] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:05.600161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:05.664451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf4f3f-bd96-4233-951c-63fdaac54d34 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:05.701558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:05.702182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:05.702457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:05.702728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:05.703482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.domain.id=default, user.id=16d94c9123ce4b32b47b3b412b38ab1b) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:05.703800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:05.705317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e392c3-2ca6-40a8-ab0b-37176bd28ffb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:05.712018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 28/56] 10.222.0.85 () {56 vars in 1322 bytes} [Wed Nov 2 12:14:05 2022] GET /identity/v3/role_assignments?user.id=16d94c9123ce4b32b47b3b412b38ab1b&scope.domain.id=default&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 530 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:05.748864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f093d86-0b14-4c6f-90dd-d834377d4174 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:05.790776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cda68-bce6-413f-8faa-210cd8b21e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:05.791456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cda68-bce6-413f-8faa-210cd8b21e60 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:05.791829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cda68-bce6-413f-8faa-210cd8b21e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:05.792212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cda68-bce6-413f-8faa-210cd8b21e60 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:05.793044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cda68-bce6-413f-8faa-210cd8b21e60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:05.794653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cda68-bce6-413f-8faa-210cd8b21e60 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:05.944303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 29/57] 10.222.0.85 () {60 vars in 1140 bytes} [Wed Nov 2 12:14:05 2022] POST /identity/v3/projects => generated 298 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:14:08.181596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a9ab-be64-46a2-ae81-72a86ccf91b4 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:08.182167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a9ab-be64-46a2-ae81-72a86ccf91b4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:08.182570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a9ab-be64-46a2-ae81-72a86ccf91b4 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:08.185075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 29/58] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:08.191031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:08.191559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:08.191559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:08.211197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:08.212401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:08.230390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:08.284764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 30/59] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:08.690525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a9ab-be64-46a2-ae81-72a86ccf91b4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:08.803796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a9ab-be64-46a2-ae81-72a86ccf91b4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:08.805284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:08.840966 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ce3de-8405-42c4-80f1-26c9c4f8ef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:08.841625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ce3de-8405-42c4-80f1-26c9c4f8ef8f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:08.841900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ce3de-8405-42c4-80f1-26c9c4f8ef8f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:08.842170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ce3de-8405-42c4-80f1-26c9c4f8ef8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:08.843292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ce3de-8405-42c4-80f1-26c9c4f8ef8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:08.844734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ce3de-8405-42c4-80f1-26c9c4f8ef8f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:08.846335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 30/60] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:08.852906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:08.977611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a687c7-5b5c-485a-aca6-3d13947e3cb9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:09.014114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118de02-5c71-4ee1-854c-da259a54611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:09.014726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118de02-5c71-4ee1-854c-da259a54611c None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:09.015003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118de02-5c71-4ee1-854c-da259a54611c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:09.015268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118de02-5c71-4ee1-854c-da259a54611c None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:09.015926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118de02-5c71-4ee1-854c-da259a54611c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:09.017386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118de02-5c71-4ee1-854c-da259a54611c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:09.045908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 31/61] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:08 2022] POST /identity/v3/users => generated 275 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:12.709853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c395ba-e7b7-4085-ad65-2f3b0b2c29c7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:12.710888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c395ba-e7b7-4085-ad65-2f3b0b2c29c7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:12.711384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c395ba-e7b7-4085-ad65-2f3b0b2c29c7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:12.713118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 31/62] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:12.719168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab5f1-6f8b-498c-8ae8-63437ed4084a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:12.720075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab5f1-6f8b-498c-8ae8-63437ed4084a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:12.720550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab5f1-6f8b-498c-8ae8-63437ed4084a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:12.745838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6b7ab5f1-6f8b-498c-8ae8-63437ed4084a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:12.747409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b7ab5f1-6f8b-498c-8ae8-63437ed4084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:12.749867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:12.855907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 32/63] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:13.001940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23adda9-30db-45c9-8690-7bb089c14b1c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.002564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23adda9-30db-45c9-8690-7bb089c14b1c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.003064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23adda9-30db-45c9-8690-7bb089c14b1c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.004761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 32/64] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:13.013829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2558af-eee9-4831-9ea7-ae90775e9ffd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.014325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2558af-eee9-4831-9ea7-ae90775e9ffd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.014666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2558af-eee9-4831-9ea7-ae90775e9ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.046452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3e2558af-eee9-4831-9ea7-ae90775e9ffd None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:13.047721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3e2558af-eee9-4831-9ea7-ae90775e9ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:13.150529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 33/65] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:13.154854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f326b-0930-4513-a949-25d348f2b8ac None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.154854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f326b-0930-4513-a949-25d348f2b8ac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.154854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f326b-0930-4513-a949-25d348f2b8ac None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.155528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 33/66] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:13.160894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56499442-c4fa-41f1-a5fd-18a2178b22ea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.161211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56499442-c4fa-41f1-a5fd-18a2178b22ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.161488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56499442-c4fa-41f1-a5fd-18a2178b22ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.185534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-56499442-c4fa-41f1-a5fd-18a2178b22ea None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:13.186806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-56499442-c4fa-41f1-a5fd-18a2178b22ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:13.263929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f326b-0930-4513-a949-25d348f2b8ac None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.280647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:13.288668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 34/67] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:13.315220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb906e7-36c5-4004-85dc-53379457ac59 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.315549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb906e7-36c5-4004-85dc-53379457ac59 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.315779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb906e7-36c5-4004-85dc-53379457ac59 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.317208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 35/68] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:14:13.322829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.323326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.323821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.378246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:13.379829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:13.399376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f326b-0930-4513-a949-25d348f2b8ac None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:13.476012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:13.477084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.477611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.478331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.490773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:13.497091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:13.497952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:13.498718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:13.499938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 34/69] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:13.500289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.502624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 36/70] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:13.512206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.557382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:13.655193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813923d-6ad6-4562-9c6b-39642e750af7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:13.688234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:13.698813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b457b7-79e6-4b3f-a05a-412d91a9639b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:13.699192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:13.699921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.700301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.700669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.701832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:13.702262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:13.704820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:13.716650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 37/71] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:13.717889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.750738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:13.751967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.752443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.752669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.764644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:13.770667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:13.771573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 35/72] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:13.773877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.840237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ee1af-b843-4c91-99d0-c92e4752f04a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:13.876560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77684eb-80d5-4a08-a3b7-1fb5f04960b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:13.877241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77684eb-80d5-4a08-a3b7-1fb5f04960b0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.877552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77684eb-80d5-4a08-a3b7-1fb5f04960b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.877797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77684eb-80d5-4a08-a3b7-1fb5f04960b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.877945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:13.889599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77684eb-80d5-4a08-a3b7-1fb5f04960b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:13.889911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e4aa2-e4ec-4b9f-ae0a-4adb4a2c267f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c77684eb-80d5-4a08-a3b7-1fb5f04960b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:13.895551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:13.896524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:13.897853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 38/73] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:13.898872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547db0-5f34-4d44-878c-5e58f10d9950 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.898872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547db0-5f34-4d44-878c-5e58f10d9950 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.898872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547db0-5f34-4d44-878c-5e58f10d9950 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.899809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 39/74] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:13.904187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc547db0-5f34-4d44-878c-5e58f10d9950 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.942883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:13.943607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:13.944118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:13.944618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:13.946676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:13.949339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:13.957035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 36/75] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:13.957035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:13.964832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.018118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc547db0-5f34-4d44-878c-5e58f10d9950 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.059112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.059732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.060015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.060288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.060968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:14.061305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.062701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.074137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 40/76] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:14.076605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d9aa-678d-4a10-9fbf-99a58e550f24 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.078598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.113464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf82be-2f1b-4d30-ab30-7a9b23121618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.114016 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf82be-2f1b-4d30-ab30-7a9b23121618 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.114290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf82be-2f1b-4d30-ab30-7a9b23121618 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.114556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf82be-2f1b-4d30-ab30-7a9b23121618 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.126077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf82be-2f1b-4d30-ab30-7a9b23121618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9ecf82be-2f1b-4d30-ab30-7a9b23121618 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:14.131947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:14.132839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 37/77] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:14.134483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.135175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.135175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.154409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:14.155635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:14.174958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.217909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa73ab-0b45-4d4d-a5b9-4563844b56ba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.231933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.241815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 38/78] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:14.242687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.257833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.258722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.258999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.259272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.259947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:14.260242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.261666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.271689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 41/79] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:14.271689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.377101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c474c-333d-4cd6-b3ef-767143688a0b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.412177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19872556-9f6b-48c5-904b-d89dfe57f9db None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.431357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.432284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.432702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.433188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.476586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.477289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.477829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.478113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.478266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.497113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.498611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.500598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 42/80] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:14.501341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.543459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.564620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.566349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.642132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2907-f341-4b9e-b6cb-1786a996a878 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.682388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.682990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.683279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.683542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.684212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:14.684376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 39/81] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:14.684511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.685234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.685920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.699848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 43/82] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:14.699848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.745524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.818386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556eb14-816b-4434-9838-10f48bd228c5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.826800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:14.838839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fb175-9e24-46d2-8101-a8c6dbda9f46 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.855479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.856141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.856410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.856680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.857920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.859414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.861087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 40/83] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:14.861966 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:14.887675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:14.889329 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:14.889850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:14.890353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:14.977064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33447f-ceea-400c-b537-ab566f7156f9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:14.977383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:14.978986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:14.996827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.017822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.018846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.018846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.020564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.021780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.023268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.024903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 41/84] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:15.026129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.099243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.107813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 44/85] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:15.108664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.146425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35924-db23-4eed-8203-112fbf4d4d19 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.187085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.188169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.188626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.189159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.191927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.194448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.197464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 42/86] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:15.198914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.225858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b225d5a-c1c7-4200-aa9b-12bda4e864c4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.268065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.268625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.268907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.269362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.270069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.270631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.272091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.273761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 45/87] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:15.274515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.327536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3409f-662b-49c6-bcc4-a09a8021fd39 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.343540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.367833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.368452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.368734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.369021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.393599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96da6e4-d20e-43c7-a852-0672b711b702 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.435556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.436171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.436449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.436722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.437416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.438870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.451112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.452599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.466278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 46/88] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:15 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:15.466278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.525619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.606243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 43/89] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:15.607432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.627103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bc465-1a65-4ada-9342-631772be9cea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.649652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.680041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.680925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.681488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.681957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.684326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.686711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.689379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 47/90] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:15.693067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.779456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed897903-d497-4f8b-9aea-16664f7cb764 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.826340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.827820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.828665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.829173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:15.830269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=16d94c9123ce4b32b47b3b412b38ab1b) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:15.830757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:15.833080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:15.842660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 44/91] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/role_assignments?user.id=16d94c9123ce4b32b47b3b412b38ab1b&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:15.843757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:15.846042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2ca96-ee82-4319-b57d-5f2496de9018 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:15.871492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:15.912464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:15.913213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:15.913567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:15.913914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:16.010777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:16.038911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:16.040373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:16.042321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f4101-1dd9-4da2-ba2f-6d07011954df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:16.166814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:16.167896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:16.168354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:16.168796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] PATH_INFO: `/v3/projects/bdbe37744de541cd818d32d27498f883` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:16.173132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] RBAC: Authorizing `identity:get_project(project_id=bdbe37744de541cd818d32d27498f883)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:16.175192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:16.178410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 45/92] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/projects/bdbe37744de541cd818d32d27498f883 => generated 312 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:16.179598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:16.329814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 48/93] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:16.331088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:16.397412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:16.427786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:16.428123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4cfb0-77c1-439b-aa63-6692e10cabdf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:16.483305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35834d-4ec6-4fe6-87f0-14bd40d80db0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:16.494495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:16.495474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:16.495984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:16.497147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:16.498139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=c93deef3b5ac4785afdd5c6bf9559b40) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:16.498674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:16.500394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:16.508023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 46/94] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:16 2022] GET /identity/v3/role_assignments?user.id=c93deef3b5ac4785afdd5c6bf9559b40&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:16.509371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:16.604029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:16.604576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:16.604856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:16.605158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:16.605907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=bdbe37744de541cd818d32d27498f883, user.id=c93deef3b5ac4785afdd5c6bf9559b40) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:16.606214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:16.607696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:16.625232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 49/95] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:16 2022] GET /identity/v3/role_assignments?user.id=c93deef3b5ac4785afdd5c6bf9559b40&scope.project.id=bdbe37744de541cd818d32d27498f883&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:16.766198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9459a5f-90cc-4365-8145-fee616fa0449 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:16.863704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30085128-9919-46cd-ac66-d45db8a2ae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:16.864781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30085128-9919-46cd-ac66-d45db8a2ae7a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:16.865311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30085128-9919-46cd-ac66-d45db8a2ae7a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:16.865812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30085128-9919-46cd-ac66-d45db8a2ae7a None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:16.871431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30085128-9919-46cd-ac66-d45db8a2ae7a None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:16.874345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30085128-9919-46cd-ac66-d45db8a2ae7a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:16.878095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 47/96] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:16 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:16.888109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:17.037519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb84c1-fad4-4adb-ac7c-3062538c22de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:17.085370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:17.086008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:17.086288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:17.086563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:17.087321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] RBAC: Adding query filter params (role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=c93deef3b5ac4785afdd5c6bf9559b40) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:17.087666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:17.089173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd22944-0c3c-4554-9efa-b74d5eaced70 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:17.096007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 50/97] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:16 2022] GET /identity/v3/role_assignments?user.id=c93deef3b5ac4785afdd5c6bf9559b40&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:18.997031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da94749-3185-49f2-bbef-463bc3b22481 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:18.997576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da94749-3185-49f2-bbef-463bc3b22481 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:18.997926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da94749-3185-49f2-bbef-463bc3b22481 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:19.000582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 48/98] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:19.019516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d286-7f78-4d7f-b795-22cdef2c93ea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:19.020497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d286-7f78-4d7f-b795-22cdef2c93ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:19.020991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d286-7f78-4d7f-b795-22cdef2c93ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:19.046893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2b33d286-7f78-4d7f-b795-22cdef2c93ea None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:19.048516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2b33d286-7f78-4d7f-b795-22cdef2c93ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:19.060613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:19.147973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 51/99] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:19.232799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d86e-190f-4084-83c2-370af856ea8a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:19.233481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d86e-190f-4084-83c2-370af856ea8a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:19.233955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d86e-190f-4084-83c2-370af856ea8a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:19.235507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 49/100] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:19.242140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252662e-ece0-431b-aab3-13ee510516ff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:19.242770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252662e-ece0-431b-aab3-13ee510516ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:19.243268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252662e-ece0-431b-aab3-13ee510516ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:19.262586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f252662e-ece0-431b-aab3-13ee510516ff None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:19.264103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f252662e-ece0-431b-aab3-13ee510516ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:19.369662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 52/101] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:19.627645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056d86e-190f-4084-83c2-370af856ea8a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:19.665950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:19.827436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd366-87f8-4c0a-b52d-bf5d557af681 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:19.828200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd366-87f8-4c0a-b52d-bf5d557af681 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:19.828897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd366-87f8-4c0a-b52d-bf5d557af681 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:19.832067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 53/102] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:19.839249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:19.840119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:19.840866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:19.895537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:19.898256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:19.899646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056d86e-190f-4084-83c2-370af856ea8a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:20.032281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:20.033456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:20.033927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:20.034452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:20.085250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:20.096060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:20.097896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:20.099400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 50/103] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:20.100708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:20.139609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 54/104] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:20.142164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:20.156557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:20.286597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:20.353057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8699-1a2e-4452-90f9-26306c5ddbef None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:20.435475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce313d-7138-4981-9aeb-1ede380281f8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:20.478369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:20.479916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:20.480733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:20.481583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:20.490419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:20.493764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:20.497301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 51/105] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:20.504184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:20.508913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:20.509576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:20.509857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:20.510131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:20.510821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:20.511122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:20.512517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:20.532433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 55/106] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:20.540348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:20.688695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:20.780457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07663f2-6c46-4b7b-9054-8ce9faa6494e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:20.787442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537d995-1e4b-4e59-b47c-cb3259052735 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:20.916552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505b9b4-e9aa-4973-a11b-a8d16ac5f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:20.919031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b9b4-e9aa-4973-a11b-a8d16ac5f63b None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:20.920674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b9b4-e9aa-4973-a11b-a8d16ac5f63b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:20.922225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b9b4-e9aa-4973-a11b-a8d16ac5f63b None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:20.924352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505b9b4-e9aa-4973-a11b-a8d16ac5f63b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:20.928175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:20.929083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:20.929418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505b9b4-e9aa-4973-a11b-a8d16ac5f63b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:20.929701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:20.930189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] PATH_INFO: `/v3/users/071d79f205a64ea785a6cf44b0d17c77` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:20.948218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] RBAC: Authorizing `identity:get_user(user_id=071d79f205a64ea785a6cf44b0d17c77)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:20.949965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:20.951897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 56/107] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:20 2022] GET /identity/v3/users/071d79f205a64ea785a6cf44b0d17c77 => generated 289 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:20.953897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:20.970539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:21.000765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 52/108] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:14:20 2022] POST /identity/v3/projects => generated 302 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:21.003521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb880e-2b99-4c3a-8519-b62e4bdcce06 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.004004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb880e-2b99-4c3a-8519-b62e4bdcce06 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.004962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb880e-2b99-4c3a-8519-b62e4bdcce06 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.013686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 53/109] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:21.013686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb880e-2b99-4c3a-8519-b62e4bdcce06 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:21.176322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2e894-57e5-41f6-bf5d-bc5817fcba93 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:21.227577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:21.242869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56ca7a-ce27-4053-b9b5-2afb6b458972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:21.243520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56ca7a-ce27-4053-b9b5-2afb6b458972 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.243864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56ca7a-ce27-4053-b9b5-2afb6b458972 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.244135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56ca7a-ce27-4053-b9b5-2afb6b458972 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.260054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb880e-2b99-4c3a-8519-b62e4bdcce06 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:21.260775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56ca7a-ce27-4053-b9b5-2afb6b458972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-aa56ca7a-ce27-4053-b9b5-2afb6b458972 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:21.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:21.269131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:21.270400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:21.273111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 57/110] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:21.275243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.275755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.276382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.300141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:21.301810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:21.332245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f19364-acf4-461b-8e3d-e5166738625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:21.333083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f19364-acf4-461b-8e3d-e5166738625c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.333368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f19364-acf4-461b-8e3d-e5166738625c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.333642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f19364-acf4-461b-8e3d-e5166738625c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.434643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 58/111] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:21.434643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:21.446607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:21.474040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f19364-acf4-461b-8e3d-e5166738625c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:21.476199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f19364-acf4-461b-8e3d-e5166738625c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:21.558993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30001c8-5f30-416f-a8ab-4d6f538c4e97 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:21.593812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:21.641920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:21.642558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.642839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.643114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.644344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:21.644654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:21.646120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13902351-0a81-4e54-8fb1-0e597ad23824 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:21.672590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 59/112] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:21.675206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4707f-f1b9-474e-992b-f2c7ac359983 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.675949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4707f-f1b9-474e-992b-f2c7ac359983 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.676434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4707f-f1b9-474e-992b-f2c7ac359983 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.678778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 60/113] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:21.682356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4707f-f1b9-474e-992b-f2c7ac359983 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:21.716224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 54/114] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:21.716224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.716224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.716224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.744755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:21.747145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:21.880985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4707f-f1b9-474e-992b-f2c7ac359983 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:21.891712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 55/115] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:21.891712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:21.923979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:21.944840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84918095-a024-4213-8776-457eb6598dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:21.946177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918095-a024-4213-8776-457eb6598dfa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.946690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918095-a024-4213-8776-457eb6598dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.947190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84918095-a024-4213-8776-457eb6598dfa None admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.949650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84918095-a024-4213-8776-457eb6598dfa None admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:21.952576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84918095-a024-4213-8776-457eb6598dfa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:21.955839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 61/116] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:21 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:21.958417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e8b9a-db2d-4377-a8d8-ec1e36ef82a8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:21.958924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e8b9a-db2d-4377-a8d8-ec1e36ef82a8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:21.959431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e8b9a-db2d-4377-a8d8-ec1e36ef82a8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:21.962506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 62/117] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:21.963999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e8b9a-db2d-4377-a8d8-ec1e36ef82a8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.033150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17121e0-a8b3-470c-b556-dd5caa5cc79d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.054870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:22.095128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.096280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.096819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.097482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.099067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:22.102477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:22.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 56/118] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:21 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:22.110964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.124428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e8b9a-db2d-4377-a8d8-ec1e36ef82a8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.166489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84b25a-2a04-4a4d-ad70-9e20471cd9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.167106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84b25a-2a04-4a4d-ad70-9e20471cd9be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.167390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84b25a-2a04-4a4d-ad70-9e20471cd9be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.167657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84b25a-2a04-4a4d-ad70-9e20471cd9be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.182604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84b25a-2a04-4a4d-ad70-9e20471cd9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7a84b25a-2a04-4a4d-ad70-9e20471cd9be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:22.189826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:22.190948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:22.191818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:22.194367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 63/119] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:22.196130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.196588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.197039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.223890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:22.225414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:22.247664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:22.256310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c43302-81f0-4a2d-b03f-9bc375ef1b3c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.302600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.303329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.303693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.303971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.305187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:22.306681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:22.308605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 57/120] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:22 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:22.313622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.321208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 64/121] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:22.321789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.367227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:22.445906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3201-c2b2-4f8f-885a-54fd79e86f18 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.461039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8aad0-64ba-417a-8e37-a731e50f8012 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.487778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.488642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.489115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.489606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.490423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:22.490864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:22.492470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:22.502713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 65/122] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:22.503864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.538346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.539123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.539443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.540001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.540804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] RBAC: Adding query filter params (role.id=f3a873ccd0d24c608bc5c01227f253ba, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=071d79f205a64ea785a6cf44b0d17c77) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:22.541234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:22.542995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:22.550497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 58/123] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:22 2022] GET /identity/v3/role_assignments?user.id=071d79f205a64ea785a6cf44b0d17c77&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=f3a873ccd0d24c608bc5c01227f253ba => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:22.551372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.609296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:22.631939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:22.642666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb1bb1-ade1-49fb-b96c-e60d220ced4a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.696605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.697682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.698199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.698997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.717311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:22.725147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:22.726086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:22.727100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 66/124] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:22.727749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.741789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c685f-6527-4b21-ba94-2d34fe350e1b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:22.798105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:22.799209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:22.799766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:22.800283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/16d94c9123ce4b32b47b3b412b38ab1b/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:22.802941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] RBAC: Authorizing `identity:create_grant(project_id=86c6007706a043bfa43e23a20cdbe653, user_id=16d94c9123ce4b32b47b3b412b38ab1b, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:22.804984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:22.823776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 59/125] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:22 2022] PUT /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/16d94c9123ce4b32b47b3b412b38ab1b/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:22.825096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:22.974768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ea179-d989-40d0-9027-cc26028b6133 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:23.071167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:23.091761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216465c8-a553-48fe-a341-42b133744315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:23.092417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216465c8-a553-48fe-a341-42b133744315 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:23.092697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216465c8-a553-48fe-a341-42b133744315 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:23.092965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216465c8-a553-48fe-a341-42b133744315 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:23.095566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216465c8-a553-48fe-a341-42b133744315 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:23.097633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216465c8-a553-48fe-a341-42b133744315 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:23.100867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 67/126] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:22 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:23.104223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216465c8-a553-48fe-a341-42b133744315 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:23.108549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb10e3-326a-4158-bc68-a11f66f22af5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:23.155874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:23.170947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:23.171966 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:23.172448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:23.172950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:23.187111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:23.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:23.195215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:23.195986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:23.199039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 60/127] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:14:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:23.200129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:23.375535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216465c8-a553-48fe-a341-42b133744315 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:23.380267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1568d42-e323-493e-963d-d78e0aef5030 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:23.430003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:23.430696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:23.430986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:23.431266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:23.431955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:23.432262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:23.433694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:23.449397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:23.450070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:23.450351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:23.450635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:23.451984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:23.453224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 68/128] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:23.453630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:23.454797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:23.455398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 61/129] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:23 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:23.461386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:23.494154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:23.722091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:23.753189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbc904-1c89-43d8-8300-c97b0a801d9c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:23.756566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e7de6-8f5d-4a62-91b6-f781b23b07ef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:23.861563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:23.864720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:23.867334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:23.868180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:23.870444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:23.873636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:23.876258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:23.877473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:23.878570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:23.881575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:23.882471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/c93deef3b5ac4785afdd5c6bf9559b40/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:23.892473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=86c6007706a043bfa43e23a20cdbe653, user_id=c93deef3b5ac4785afdd5c6bf9559b40, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:23.895620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:23.906334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 69/130] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:14:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:23.912027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:23.934270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 62/131] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:23 2022] PUT /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/c93deef3b5ac4785afdd5c6bf9559b40/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:14:23.939136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:24.208194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:24.247075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534d85-3214-47d1-bee7-70476c8097f0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:24.258540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5de41-cd17-42a8-9e57-ddd400d495b0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:24.320221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:24.321333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:24.321718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:24.322125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:24.324267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:24.326245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:24.328527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 70/132] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:23 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:24.348313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:24.357114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94849e6a-de55-42ee-863f-e07153aa6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:24.358845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94849e6a-de55-42ee-863f-e07153aa6644 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:24.359724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94849e6a-de55-42ee-863f-e07153aa6644 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:24.360510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94849e6a-de55-42ee-863f-e07153aa6644 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:24.366498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94849e6a-de55-42ee-863f-e07153aa6644 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:24.370459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94849e6a-de55-42ee-863f-e07153aa6644 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:24.374248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 63/133] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:23 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:24.387504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3080a-b6d6-4d0b-ae7d-ee4d88656d56 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:24.389332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3080a-b6d6-4d0b-ae7d-ee4d88656d56 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:24.390272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3080a-b6d6-4d0b-ae7d-ee4d88656d56 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:24.393519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 64/134] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:24.395496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3080a-b6d6-4d0b-ae7d-ee4d88656d56 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:24.402395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:24.613724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a4fb-9a3e-4689-9c16-b0547c98ee98 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:24.670443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3080a-b6d6-4d0b-ae7d-ee4d88656d56 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:24.697331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2f30d-3ce2-427f-be3a-b1d7cf69de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:24.697987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f30d-3ce2-427f-be3a-b1d7cf69de0e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:24.698274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f30d-3ce2-427f-be3a-b1d7cf69de0e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:24.698559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f30d-3ce2-427f-be3a-b1d7cf69de0e None admin] PATH_INFO: `/v3/projects/bdbe37744de541cd818d32d27498f883` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:24.702602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f30d-3ce2-427f-be3a-b1d7cf69de0e None admin] RBAC: Authorizing `identity:get_project(project_id=bdbe37744de541cd818d32d27498f883)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:24.704100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f30d-3ce2-427f-be3a-b1d7cf69de0e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:24.705941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 71/135] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:24 2022] GET /identity/v3/projects/bdbe37744de541cd818d32d27498f883 => generated 312 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:24.707144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:24.709528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:24.710813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:24.743743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:24.744297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:24.744679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:24.744961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:24.750558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:24.751716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:24.753030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:24.754401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:24.756200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 65/136] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:24 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:24.756987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:24.786564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:24.905139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 72/137] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:24.906224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:24.951913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:25.010921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e5e6-caea-4fa5-865e-b61ac5ff0d3a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:25.085805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:25.086509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:25.086794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:25.087075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] PATH_INFO: `/v3/projects/bdbe37744de541cd818d32d27498f883/users/c93deef3b5ac4785afdd5c6bf9559b40/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:25.093032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bdbe37744de541cd818d32d27498f883, user_id=c93deef3b5ac4785afdd5c6bf9559b40, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:25.093032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:25.116946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 66/138] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:24 2022] PUT /identity/v3/projects/bdbe37744de541cd818d32d27498f883/users/c93deef3b5ac4785afdd5c6bf9559b40/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:25.203623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f8f6-c7b5-48cc-81b3-be12249e7cae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:25.308061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:25.309721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:25.310484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:25.311246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/c93deef3b5ac4785afdd5c6bf9559b40/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:25.318766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] RBAC: Authorizing `identity:create_grant(project_id=86c6007706a043bfa43e23a20cdbe653, user_id=c93deef3b5ac4785afdd5c6bf9559b40, role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:25.321996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:25.338543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:25.354669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 73/139] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:24 2022] PUT /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/c93deef3b5ac4785afdd5c6bf9559b40/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:25.436342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:25.484112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651c1f4-517c-48c2-bafd-c2b0f4d51ff7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:25.540725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d017cc-1df2-495b-88d8-6e06f8465b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:25.541663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d017cc-1df2-495b-88d8-6e06f8465b51 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:25.541952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d017cc-1df2-495b-88d8-6e06f8465b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:25.542398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d017cc-1df2-495b-88d8-6e06f8465b51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:25.543987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d017cc-1df2-495b-88d8-6e06f8465b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:25.545628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d017cc-1df2-495b-88d8-6e06f8465b51 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:25.547435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 67/140] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:25.554178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:25.695471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:25.750943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504159e-d69c-4560-966a-1a4e6bea643a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:25.821163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e1ca0-8a08-4d7c-9068-40f1547b9b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:25.822234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e1ca0-8a08-4d7c-9068-40f1547b9b8c None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:25.822727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e1ca0-8a08-4d7c-9068-40f1547b9b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:25.823170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e1ca0-8a08-4d7c-9068-40f1547b9b8c None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:25.824307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e1ca0-8a08-4d7c-9068-40f1547b9b8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:25.826887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e1ca0-8a08-4d7c-9068-40f1547b9b8c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:25.871918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 74/141] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:25 2022] POST /identity/v3/users => generated 283 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:26.130204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1da9cf-ed0e-4f3d-87f8-885aaae9afdb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:26.130648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1da9cf-ed0e-4f3d-87f8-885aaae9afdb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:26.131133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1da9cf-ed0e-4f3d-87f8-885aaae9afdb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:26.133843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 68/142] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:26.142526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:26.143074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:26.143544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:26.191313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:26.193569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:26.221581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e985d7b-48f2-420e-bd53-c2d9efe89bac None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:26.222056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e985d7b-48f2-420e-bd53-c2d9efe89bac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:26.222555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e985d7b-48f2-420e-bd53-c2d9efe89bac None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:26.225283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 69/143] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:26.232349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:26.232855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:26.233377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:26.290930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:26.292819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:26.453833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 75/144] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:26.517081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 70/145] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Nov 02 12:14:27.044227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:27.071590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:27.240526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:27.284785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:27.335764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c8fa4-8d67-4ab1-be41-fa76aa75dd8d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:27.359543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1317c-5643-4c47-b868-280090b6137d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:27.418078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:27.419159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:27.419640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:27.420122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:27.460406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:27.461470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:27.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:27.462409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:27.468666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:27.477930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:27.479429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:27.480927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:27.489907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 76/146] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:27.503152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:27.506934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:27.534480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:27.538696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:27.541893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 71/147] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:27.549426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:27.818888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbc386-1cf9-4825-87be-10fd4a597f0b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:27.889818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:27.898503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8545b3-8c5e-4774-94b5-62d332bd34ec None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:27.933724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:27.935224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:27.935977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:27.936731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:27.938251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:27.939067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:27.942513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6402355-26ac-4b5d-984f-4edf8b865176 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:27.973334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 77/148] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:27.977975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db3f73-17c8-4d4f-8e62-e2bf43ffde44 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:27.978861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db3f73-17c8-4d4f-8e62-e2bf43ffde44 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:27.979596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db3f73-17c8-4d4f-8e62-e2bf43ffde44 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:27.986943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 78/149] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:27.991541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db3f73-17c8-4d4f-8e62-e2bf43ffde44 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:27.993590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:28.022396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:28.023429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:28.023909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:28.024390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:28.025643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:28.026192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:28.028838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36604855-5833-46cb-bc8e-907895a7fb72 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:28.047054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 72/150] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:28.049877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:28.050306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:28.050785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:28.089848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:28.091784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:28.278143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 73/151] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:28.279602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:28.319483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db3f73-17c8-4d4f-8e62-e2bf43ffde44 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:28.417973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:28.418935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:28.419418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:28.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] PATH_INFO: `/v3/users/071d79f205a64ea785a6cf44b0d17c77` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:28.424294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] RBAC: Authorizing `identity:get_user(user_id=071d79f205a64ea785a6cf44b0d17c77)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:28.426981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:28.430499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 79/152] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:27 2022] GET /identity/v3/users/071d79f205a64ea785a6cf44b0d17c77 => generated 289 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:28.440026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:28.594854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:28.636708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a2ce-0de1-4bdb-97a4-a69e14e2beb0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:28.704072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:28.728658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d16b-8b40-4a73-bc3d-15d124cc5434 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:28.762169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:28.763547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:28.764249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:28.764898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] PATH_INFO: `/v3/users/16d94c9123ce4b32b47b3b412b38ab1b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:28.767215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] RBAC: Authorizing `identity:get_user(user_id=16d94c9123ce4b32b47b3b412b38ab1b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:28.770210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:28.773487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 74/153] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:28 2022] GET /identity/v3/users/16d94c9123ce4b32b47b3b412b38ab1b => generated 247 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:28.797896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:28.831992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:28.833272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:28.833771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:28.834260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:28.836434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:28.841221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:28.846805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 80/154] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:28 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:28.856532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:29.093845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daa222-f6a5-4784-a757-34502acc2690 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:29.165461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:29.166971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:29.167712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:29.168460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:29.195056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76e8a1-d2ed-4b88-b2f9-1a784afd0abf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:29.251236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:29.305776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:29.307103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:29.307577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:29.308124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/071d79f205a64ea785a6cf44b0d17c77/roles/f3a873ccd0d24c608bc5c01227f253ba` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:29.311777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] RBAC: Authorizing `identity:create_grant(project_id=86c6007706a043bfa43e23a20cdbe653, user_id=071d79f205a64ea785a6cf44b0d17c77, role_id=f3a873ccd0d24c608bc5c01227f253ba)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:29.318946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:29.353903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 81/155] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:28 2022] PUT /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653/users/071d79f205a64ea785a6cf44b0d17c77/roles/f3a873ccd0d24c608bc5c01227f253ba => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:14:29.357650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:29.402360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:29.405146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:29.628280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 75/156] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 831 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:14:29.636644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:29.685540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:29.960984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473479-6877-492a-b956-75fbc8030afd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:29.967457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:29.999983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83f04d-e9ab-4c51-abfa-f17623cd3a12 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:30.089185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:30.090484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:30.090985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:30.091411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:30.091884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:30.092914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:30.093426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:30.093897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:30.097509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:30.100259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:30.103592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 82/157] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:29 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:30.113952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:30.115896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:30.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:30.127860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:30.129051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 76/158] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:30.137908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:30.274130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:30.412891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb748-4e23-435e-b42d-d37ddeab27da None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:30.450148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e732a5-e17f-4f44-86af-67313b15d508 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:30.535433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:30.538841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:30.539595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:30.540671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:30.543129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:30.545250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:30.548315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:30.595773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 77/159] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:30.602241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:30.603647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:30.604398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:30.605023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:30.605206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:30.606682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=16d94c9123ce4b32b47b3b412b38ab1b) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:30.607512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:30.610259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:30.627121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 83/160] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:30 2022] GET /identity/v3/role_assignments?user.id=16d94c9123ce4b32b47b3b412b38ab1b&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:30.639128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:30.866237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a7ad-fe97-4c57-8bb7-83ca2884ca1e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:30.947247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:30.948296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:30.948765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:30.949258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:30.954441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:30.957832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:30.961071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 78/161] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:30 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:30.971146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:31.221080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a02ed3-3a55-4bd6-9536-27a4720ac5c7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:31.245458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:31.294101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:31.295245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:31.295738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:31.296207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:31.457257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:31.460382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:31.715351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 84/162] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:31.725062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:31.915152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:31.960037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ed2e-04bb-4de8-88fc-3ddb68b584ef None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:32.026955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e7c9d-aeaa-484d-92e1-ff083a779a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:32.028040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e7c9d-aeaa-484d-92e1-ff083a779a5f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:32.028529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e7c9d-aeaa-484d-92e1-ff083a779a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:32.029024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e7c9d-aeaa-484d-92e1-ff083a779a5f None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:32.031457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e7c9d-aeaa-484d-92e1-ff083a779a5f None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:32.034901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e7c9d-aeaa-484d-92e1-ff083a779a5f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:32.038348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 79/163] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:31 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:32.048048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:32.070598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:32.115877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c273-af79-4fc5-86b6-34321827e6ba None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:32.116364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c273-af79-4fc5-86b6-34321827e6ba None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:32.116718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c273-af79-4fc5-86b6-34321827e6ba None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:32.119485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 80/164] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:32.126078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ab38e-795b-43b0-bfb0-030534e18499 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:32.126609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ab38e-795b-43b0-bfb0-030534e18499 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:32.127094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ab38e-795b-43b0-bfb0-030534e18499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:32.182899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a97ab38e-795b-43b0-bfb0-030534e18499 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:32.184824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a97ab38e-795b-43b0-bfb0-030534e18499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:32.322988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136636b0-238e-4521-a81c-433ca8ae7b2f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:32.433959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:32.435248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:32.437879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:32.438404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:32.440403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=c93deef3b5ac4785afdd5c6bf9559b40) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:32.443107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:32.448020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5916d-a3d9-469b-84c4-3df22031a869 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:32.452076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 81/165] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:32.456209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d37650-1ed2-4085-a09e-9d1a6de459f8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:32.459398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d37650-1ed2-4085-a09e-9d1a6de459f8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:32.460140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d37650-1ed2-4085-a09e-9d1a6de459f8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:32.465294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 82/166] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:32.470117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 85/167] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:32 2022] GET /identity/v3/role_assignments?user.id=c93deef3b5ac4785afdd5c6bf9559b40&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 608 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:32.482209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feddbb3-8cf1-4881-8572-2396ead691ab None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:32.483027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feddbb3-8cf1-4881-8572-2396ead691ab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:32.483604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feddbb3-8cf1-4881-8572-2396ead691ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:32.521777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9feddbb3-8cf1-4881-8572-2396ead691ab None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:32.524179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9feddbb3-8cf1-4881-8572-2396ead691ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:32.695202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 83/168] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:33.038100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768257f-89a8-48d2-806c-460f348b5b71 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:33.039121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768257f-89a8-48d2-806c-460f348b5b71 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:33.039901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768257f-89a8-48d2-806c-460f348b5b71 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:33.044337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 86/169] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:33.051536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:33.052354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:33.052486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:33.081087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:33.094720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:33.097395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:33.171849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768257f-89a8-48d2-806c-460f348b5b71 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:33.198413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 84/170] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:33.349173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:33.406134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768257f-89a8-48d2-806c-460f348b5b71 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:33.479107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:33.489624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:33.491673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:33.492412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:33.493200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:33.520596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:33.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:33.538464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:33.540053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:33.548784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 87/171] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:33.558642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:33.773508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fe89a-3c5f-47c3-ae5d-922c33c987ef None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:33.849854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:33.850787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:33.851070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:33.851341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:33.861956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:33.882089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:33.887167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:33.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:33.890406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 85/172] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:14:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:33.898135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:33.934484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40901a-2ab3-4009-87a1-d2b0ec8175a5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.018838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:34.023629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.025293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.030280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.032109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:34.033070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:34.036749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:34.054560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 88/173] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:34.055055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:34.061844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:34.207232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268840df-4201-4229-84b1-7150637e1d11 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.258292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:34.258682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a8e98-bc6c-45a8-be65-b5a09c8c9660 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.259515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.259515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.259685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.260513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:34.260751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:34.262145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:34.273051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 86/174] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:14:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:34.273971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:34.310042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40285b5e-3780-4422-a51b-c48c03b87d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:34.310693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285b5e-3780-4422-a51b-c48c03b87d36 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.310997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285b5e-3780-4422-a51b-c48c03b87d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.311237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285b5e-3780-4422-a51b-c48c03b87d36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.333340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40285b5e-3780-4422-a51b-c48c03b87d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-40285b5e-3780-4422-a51b-c48c03b87d36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:34.343498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:34.344650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:34.345958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:34.347030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 89/175] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:34.347949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18d786-1730-4ce1-8d48-8f25a1c4b71b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.348310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18d786-1730-4ce1-8d48-8f25a1c4b71b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.348589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18d786-1730-4ce1-8d48-8f25a1c4b71b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.350381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 90/176] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:34.354768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c36d0f-55e9-4c2c-9429-c405d250421e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.355219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c36d0f-55e9-4c2c-9429-c405d250421e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.355506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c36d0f-55e9-4c2c-9429-c405d250421e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.357824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 91/177] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:34.358734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c36d0f-55e9-4c2c-9429-c405d250421e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:34.382527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:34.451870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:34.464180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d53a68-822b-4d1f-858c-f2690f0f3989 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.514883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:34.515635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.515913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.516181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.517995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:34.519490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:34.521351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 87/178] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:34 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:34.526655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:34.535528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c36d0f-55e9-4c2c-9429-c405d250421e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.589520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b54039-8252-4608-964c-5562527efc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:34.590159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54039-8252-4608-964c-5562527efc77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.590455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54039-8252-4608-964c-5562527efc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.590706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b54039-8252-4608-964c-5562527efc77 None admin] PATH_INFO: `/v3/users/c93deef3b5ac4785afdd5c6bf9559b40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.592388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b54039-8252-4608-964c-5562527efc77 None admin] RBAC: Authorizing `identity:get_user(user_id=c93deef3b5ac4785afdd5c6bf9559b40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:34.594209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b54039-8252-4608-964c-5562527efc77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:34.596080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 92/179] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:34 2022] GET /identity/v3/users/c93deef3b5ac4785afdd5c6bf9559b40 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:34.597306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7972a96-21f8-40f6-8744-f93a8b137995 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.597582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7972a96-21f8-40f6-8744-f93a8b137995 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.597851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7972a96-21f8-40f6-8744-f93a8b137995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.618561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e7972a96-21f8-40f6-8744-f93a8b137995 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:34.620390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e7972a96-21f8-40f6-8744-f93a8b137995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:34.665245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e8dd9-a4cf-462c-a2c4-f6aa2df2039e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.700546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:34.707546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 93/180] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:34.710012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.710505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.711001 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.728690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:34.730250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:34.731038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:34.731775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:34.733587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:34.734415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:34.738141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:34.747486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:34.749793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:34.758453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 88/181] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:34.767020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:34.838934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:34.864116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 94/182] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:34.865054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:34.986161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598e5de-5855-4418-83ca-2bb00c50d5ea None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:34.994408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee1cbc-36b6-4937-807e-d64736078385 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:35.025239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.025892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.026163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.026440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.065893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d95dc-ce75-495d-ad24-feb6e68d4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.067800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d95dc-ce75-495d-ad24-feb6e68d4e6b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.068647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d95dc-ce75-495d-ad24-feb6e68d4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.069570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d95dc-ce75-495d-ad24-feb6e68d4e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.088187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.107815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:35.109520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:35.185581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.207847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d95dc-ce75-495d-ad24-feb6e68d4e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:35.210777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d95dc-ce75-495d-ad24-feb6e68d4e6b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:35.227024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 95/183] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:35.228412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.336043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.356405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb71e-7bb2-45e7-a76d-a7e43b7133ba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:35.367096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 89/184] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:35.369250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4bb4c-0728-4db7-a493-6f392fb24297 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.369732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4bb4c-0728-4db7-a493-6f392fb24297 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.370212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4bb4c-0728-4db7-a493-6f392fb24297 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.372899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 90/185] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:35.374228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4bb4c-0728-4db7-a493-6f392fb24297 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.398957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.399614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.399890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.400159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] PATH_INFO: `/v3/users/9d132112663d4c9285227f4fbccc4a94` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.416790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] RBAC: Authorizing `identity:get_user(user_id=9d132112663d4c9285227f4fbccc4a94)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:35.418344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:35.420081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 96/186] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:35 2022] GET /identity/v3/users/9d132112663d4c9285227f4fbccc4a94 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:35.421114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.515247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.536845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4bb4c-0728-4db7-a493-6f392fb24297 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:35.564582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943cd29-99b9-4b97-8710-3c5250e14468 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:35.597534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.598352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.598746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.599188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.616637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:35.620751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.621708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.622167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.622614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.624636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:35.625882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:35.627664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:35.629239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:35.630714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:35.630839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 91/187] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:14:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:35.630839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.631074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 97/188] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:35.632943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.650314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.787118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a2fc7-2782-414d-9b51-7dbe88e4d2eb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:35.790902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c37eff-be53-48e5-9372-7ec51a45cf83 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:35.826932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1791fae-eae0-44d5-a0a3-33c07ee784f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.827883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1791fae-eae0-44d5-a0a3-33c07ee784f9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.828158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1791fae-eae0-44d5-a0a3-33c07ee784f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.828438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1791fae-eae0-44d5-a0a3-33c07ee784f9 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.830397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1791fae-eae0-44d5-a0a3-33c07ee784f9 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:35.831890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1791fae-eae0-44d5-a0a3-33c07ee784f9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:35.833649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 92/189] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:35 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:35.842156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.842156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.842156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.842536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:35.842536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:35.842536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:35.842536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] PATH_INFO: `/v3/projects/bdbe37744de541cd818d32d27498f883` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:35.843843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] RBAC: Authorizing `identity:get_project(project_id=bdbe37744de541cd818d32d27498f883)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:35.845439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:35.847231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 98/190] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:35 2022] GET /identity/v3/projects/bdbe37744de541cd818d32d27498f883 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:35.848062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.856423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:35.857748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:35.873247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.950245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:35.955159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 93/191] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:35.956633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:35.990429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06e3b-c4ae-48b3-90bd-9188b11ca0bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:36.032525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:36.033183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:36.033454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:36.033722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:36.122955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:36.125492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:36.154512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267681b-d931-4478-9fdf-353fb92c2e3e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:36.215433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:36.217523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:36.217977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:36.218336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:36.219133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:36.221455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:36.265042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 94/192] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:35 2022] POST /identity/v3/users => generated 297 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:36.266151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:36.270952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:36.276430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:36.314757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 99/193] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:36.316276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:36.444142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790269-295a-4ee4-8a59-57621f61efd4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:36.488172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee049820-83a5-41c4-b97b-0c77c277759f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:36.499971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:36.500937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:36.501428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:36.501811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:36.502635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:36.503132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:36.505051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:36.524005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 95/194] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:14:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:36.524005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:36.542128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:36.542754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:36.543031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:36.543305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:36.544536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] RBAC: Adding query filter params (role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=c93deef3b5ac4785afdd5c6bf9559b40) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:36.544844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:36.546337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:36.555484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 100/195] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:36 2022] GET /identity/v3/role_assignments?user.id=c93deef3b5ac4785afdd5c6bf9559b40&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:36.556649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:36.643065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:36.658854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:36.745481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e9b2-163c-4359-9afc-f4cdfee87b36 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:36.763717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bde6d-4a07-4906-be61-dfe7a2823e77 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:36.813597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:36.815014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:36.815944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:36.816684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:36.817893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:36.818543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:36.818821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:36.819095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:36.819457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=bdbe37744de541cd818d32d27498f883, user.id=c93deef3b5ac4785afdd5c6bf9559b40) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:36.820194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:36.820344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:36.821955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:36.823357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:36.824012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 101/196] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:36 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:36.825791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:36.838430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 96/197] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:36 2022] GET /identity/v3/role_assignments?user.id=c93deef3b5ac4785afdd5c6bf9559b40&scope.project.id=bdbe37744de541cd818d32d27498f883&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:36.840375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:37.089140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046df0d-5267-4377-b4cf-9d3e2d243707 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:37.119553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb9580-c35f-4fdf-b0fc-6ea1962eef70 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:37.135211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:37.146537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:37.165947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:37.167254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:37.167725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:37.168359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] PATH_INFO: `/v3/users/9d132112663d4c9285227f4fbccc4a94` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:37.170775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] RBAC: Authorizing `identity:get_user(user_id=9d132112663d4c9285227f4fbccc4a94)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:37.171879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:37.173483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:37.173966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:37.174433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:37.175401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:37.178468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 97/198] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:36 2022] GET /identity/v3/users/9d132112663d4c9285227f4fbccc4a94 => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:37.180205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:37.205472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:37.218611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:37.221065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 102/199] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:37.223498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:37.375013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebbdff-072c-4c2a-bd63-5853d3b1c5e4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:37.409913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedc856-bb4c-4496-97c9-f9ac01be8f0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:37.450722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:37.451392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:37.451678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:37.451954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:37.458047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:37.459557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:37.460312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:37.461103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:37.463102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=9d132112663d4c9285227f4fbccc4a94) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:37.464010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:37.472390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:37.484637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 98/200] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:37 2022] GET /identity/v3/role_assignments?user.id=9d132112663d4c9285227f4fbccc4a94&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:37.487493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:37.504407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:37.541781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:37.543437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:37.617532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:37.683416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7fbc-f9f2-478e-b4c9-2e6b1bee5642 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:37.761048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 103/201] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:37.765104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:37.792704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:37.794267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:37.795012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:37.795916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:37.797660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:37.798618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:37.801707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:37.839092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 99/202] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:37.849211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:37.950860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:38.011744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ff9ed-4acd-4129-a64c-aa3f2a1f1394 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:38.080186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:38.081653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:38.086059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:38.089238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:38.093062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:38.096304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:38.102760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 104/203] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:37 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:38.114714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:38.161820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bc52f-ee9e-4c4c-9db4-130e25675b07 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:38.252663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:38.281259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:38.282736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:38.283422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:38.284136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] PATH_INFO: `/v3/users/071d79f205a64ea785a6cf44b0d17c77` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:38.292230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] RBAC: Authorizing `identity:get_user(user_id=071d79f205a64ea785a6cf44b0d17c77)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:38.292230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:38.293544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 100/204] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:37 2022] GET /identity/v3/users/071d79f205a64ea785a6cf44b0d17c77 => generated 289 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:38.299708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:38.395971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d52b9-1392-4f02-bdac-f32f7f4bacb2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:38.462850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:38.463467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:38.463745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:38.464006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:38.464754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] RBAC: Adding query filter params (role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=9d132112663d4c9285227f4fbccc4a94) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:38.465082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:38.469083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:38.482550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 105/205] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:38 2022] GET /identity/v3/role_assignments?user.id=9d132112663d4c9285227f4fbccc4a94&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:38.491157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dffa4-62d7-431f-8dcd-e34823462b8a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:38.563748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:38.564700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:38.565217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:38.565704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:38.727471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:38.750451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:38.753887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:38.961559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 101/206] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:38.967428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:38.975443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:39.142923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa29b-0d5f-470e-bc7c-b587714d5e02 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:39.199125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e6714-ba28-436c-a7c1-9dc67ec3581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:39.199779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6714-ba28-436c-a7c1-9dc67ec3581e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:39.200055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6714-ba28-436c-a7c1-9dc67ec3581e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:39.200324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6714-ba28-436c-a7c1-9dc67ec3581e None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:39.201551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e6714-ba28-436c-a7c1-9dc67ec3581e None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:39.203054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e6714-ba28-436c-a7c1-9dc67ec3581e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:39.205590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 106/207] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:38 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:39.211728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:39.350579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:39.369251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d60857-1cc8-4321-b716-2c565291f55e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:39.420558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:39.421244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:39.421519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:39.421787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:39.422570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] RBAC: Adding query filter params (role.id=f3a873ccd0d24c608bc5c01227f253ba, scope.project.id=86c6007706a043bfa43e23a20cdbe653, user.id=071d79f205a64ea785a6cf44b0d17c77) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:39.422872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:39.424336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4539-452a-4f8b-9ebd-04152f9fa9c7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:39.431424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 102/208] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:39 2022] GET /identity/v3/role_assignments?user.id=071d79f205a64ea785a6cf44b0d17c77&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=f3a873ccd0d24c608bc5c01227f253ba => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:40.027460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86825d-5201-48ce-b51a-7f1ef46e3088 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.028284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86825d-5201-48ce-b51a-7f1ef46e3088 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.028284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86825d-5201-48ce-b51a-7f1ef46e3088 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.029827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 107/209] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:40.036887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5803-79db-46e2-a9f0-bfc3328b0254 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.037719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5803-79db-46e2-a9f0-bfc3328b0254 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.037719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5803-79db-46e2-a9f0-bfc3328b0254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.060263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0e9c5803-79db-46e2-a9f0-bfc3328b0254 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:40.061526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e9c5803-79db-46e2-a9f0-bfc3328b0254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:40.150368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 103/210] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:40.370260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99f557-e1e7-4c0f-9f83-74ecf4e175f2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.370492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99f557-e1e7-4c0f-9f83-74ecf4e175f2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.370913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99f557-e1e7-4c0f-9f83-74ecf4e175f2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.372051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 108/211] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:40.378056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.378323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.378639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.399220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:40.400430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:40.459391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99f557-e1e7-4c0f-9f83-74ecf4e175f2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:40.498927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 104/212] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:40.579357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:40.640200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99f557-e1e7-4c0f-9f83-74ecf4e175f2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:40.705430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:40.706413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.706886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.707352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.728728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:40.738945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:40.740679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:40.743049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 109/213] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:40.750152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:40.779367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:40.831962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:40.933056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52792-3468-4f19-b8cd-ee3d0dc7d047 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:40.975264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:40.976137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.976410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.976676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.977383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:40.977680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:40.979101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396088cc-9683-445c-a27a-bc61bb8a5fd1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:40.988687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 105/214] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:40.990132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417246e-bfdf-4f84-83de-c6d7c4402dd8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:40.990423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417246e-bfdf-4f84-83de-c6d7c4402dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:40.990696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417246e-bfdf-4f84-83de-c6d7c4402dd8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:40.991953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 106/215] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:40.997062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417246e-bfdf-4f84-83de-c6d7c4402dd8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:41.038712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc451e-8a26-4dd9-8148-6104aa79c60e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:41.076796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:41.088246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f761b-788a-44a0-9c60-4cec419a6cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:41.089116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f761b-788a-44a0-9c60-4cec419a6cfe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:41.089626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f761b-788a-44a0-9c60-4cec419a6cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:41.090037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f761b-788a-44a0-9c60-4cec419a6cfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:41.091478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f761b-788a-44a0-9c60-4cec419a6cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:41.092323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:41.093888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f761b-788a-44a0-9c60-4cec419a6cfe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:41.096640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 110/216] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:41.100094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:41.100846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:41.101358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:41.125492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:41.126268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417246e-bfdf-4f84-83de-c6d7c4402dd8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:41.127382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:41.174505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:41.175468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:41.175973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:41.176360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] PATH_INFO: `/v3/users/62c5e70709924f08a56f4a90c9dc9ea3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:41.195566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] RBAC: Authorizing `identity:get_user(user_id=62c5e70709924f08a56f4a90c9dc9ea3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:41.197364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:41.199930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 107/217] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:40 2022] GET /identity/v3/users/62c5e70709924f08a56f4a90c9dc9ea3 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:41.206369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:41.214914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 111/218] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:41.215615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:41.386010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af509d-f794-4709-9488-80d5ef3bd659 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:41.399545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829355d0-ddaf-44c4-ad41-9d0aa731b254 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:41.448754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:41.450420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:41.451141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:41.451856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:41.452793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:41.455044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:41.459371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:41.465099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:41.466043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:41.466504 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:41.466957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:41.479545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:41.516968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 108/219] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:41 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:41.516968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:41.584720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:41.586425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:41.746882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f9b4b-441c-49e3-8766-1b1509b7c948 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:41.747119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 112/220] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:41.753151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:41.807281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:41.809565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:41.809844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:41.810283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:41.810821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:41.835112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:41.848056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:41.849880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:41.851650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 109/221] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:41.861666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:41.961453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdd701-f29e-4db6-8abc-9846db9db201 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:41.963551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:42.034071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de190b3-c67e-4836-bd3b-2f891514b64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:42.036436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de190b3-c67e-4836-bd3b-2f891514b64b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.037145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de190b3-c67e-4836-bd3b-2f891514b64b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.037815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de190b3-c67e-4836-bd3b-2f891514b64b None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.040577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de190b3-c67e-4836-bd3b-2f891514b64b None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:42.049024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de190b3-c67e-4836-bd3b-2f891514b64b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:42.049750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 113/222] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:41 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:42.051802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba605b-dc60-4cf4-aa5e-39f48ff6e63d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.052309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba605b-dc60-4cf4-aa5e-39f48ff6e63d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.052756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba605b-dc60-4cf4-aa5e-39f48ff6e63d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.056610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 114/223] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:42.061776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b32b08-fcc5-4daa-b0f8-498c219d9600 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:42.063539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba605b-dc60-4cf4-aa5e-39f48ff6e63d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:42.115463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:42.118193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.119142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.120252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.122163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:42.122921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:42.125789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8f936-7bce-45b4-be4c-60337ba2e225 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:42.146622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 110/224] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:42.150058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.151005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.151682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.190340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:42.194126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:42.254792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:42.268929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba605b-dc60-4cf4-aa5e-39f48ff6e63d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:42.337093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:42.337520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 111/225] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:42.338811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.339604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.340355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.341973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=62c5e70709924f08a56f4a90c9dc9ea3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:42.342638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:42.343125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:42.346941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:42.370255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 115/226] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:42 2022] GET /identity/v3/role_assignments?user.id=62c5e70709924f08a56f4a90c9dc9ea3&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:42.460105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:42.604307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8da22a-c7a1-4bb1-bb23-5134083c4a77 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:42.658137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:42.658796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.659077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.659347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] PATH_INFO: `/v3/users/9d132112663d4c9285227f4fbccc4a94` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.660580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] RBAC: Authorizing `identity:get_user(user_id=9d132112663d4c9285227f4fbccc4a94)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:42.662093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:42.663923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 112/227] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:42 2022] GET /identity/v3/users/9d132112663d4c9285227f4fbccc4a94 => generated 283 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:42.670243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:42.805901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40158a4-4cb1-48aa-ad9b-3e3fd2184abc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:42.853907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:42.854610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:42.854839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:42.855112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:42.859200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:42.860687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:42.863065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 116/228] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:42 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:42.869136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:42.929426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:42.984739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:43.048514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9f18-4ec1-47fb-9bb6-6db7c73f6db7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:43.070189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:43.097711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:43.098770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:43.099251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:43.099741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/9d132112663d4c9285227f4fbccc4a94/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:43.102551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] RBAC: Authorizing `identity:create_grant(project_id=1950f1d110d94df48b26b19d33e527ed, user_id=9d132112663d4c9285227f4fbccc4a94, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:43.104453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:43.121552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 113/229] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:42 2022] PUT /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/9d132112663d4c9285227f4fbccc4a94/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:43.160765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010c555-2fca-4dbc-9af4-92cae44d18e1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:43.212948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:43.213680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:43.213963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:43.214238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:43.233286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:43.239797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:43.240778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:43.241642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 117/230] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:14:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:43.250771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:43.420689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a53949-fb94-4ab2-9a57-c59392f45421 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:43.470959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:43.471711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:43.471990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:43.472261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:43.472956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:43.473288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:43.474724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:43.490380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 114/231] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:14:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:43.498986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:43.614642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:43.640745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ace73-c309-4e66-b311-90c30e069f50 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:43.706807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:43.707475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:43.707757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:43.708084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] PATH_INFO: `/v3/users/9d132112663d4c9285227f4fbccc4a94` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:43.713938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] RBAC: Authorizing `identity:get_user(user_id=9d132112663d4c9285227f4fbccc4a94)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:43.715417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:43.717513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 118/232] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:43 2022] GET /identity/v3/users/9d132112663d4c9285227f4fbccc4a94 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:43.724264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:43.788979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:43.883965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a863b-caff-4a61-9ae7-51629cda1165 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:43.935442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddde9bb-bf3c-483a-942f-e96bb5f1806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:43.936113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde9bb-bf3c-483a-942f-e96bb5f1806f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:43.936389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde9bb-bf3c-483a-942f-e96bb5f1806f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:43.936663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddde9bb-bf3c-483a-942f-e96bb5f1806f None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:43.938009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde9bb-bf3c-483a-942f-e96bb5f1806f None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:43.939522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddde9bb-bf3c-483a-942f-e96bb5f1806f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:43.941390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 115/233] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:43 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:43.947559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:44.090729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35953b-c3e6-485d-88b2-8fdc08a90881 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:44.128820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78fa9f-2f40-41a9-8a0e-d0bf4087cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:44.129533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78fa9f-2f40-41a9-8a0e-d0bf4087cfdb None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:44.130092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78fa9f-2f40-41a9-8a0e-d0bf4087cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:44.130362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78fa9f-2f40-41a9-8a0e-d0bf4087cfdb None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/9d132112663d4c9285227f4fbccc4a94/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:44.132609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78fa9f-2f40-41a9-8a0e-d0bf4087cfdb None admin] RBAC: Authorizing `identity:create_grant(project_id=1950f1d110d94df48b26b19d33e527ed, user_id=9d132112663d4c9285227f4fbccc4a94, role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:44.134101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78fa9f-2f40-41a9-8a0e-d0bf4087cfdb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:44.155758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 119/234] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:43 2022] PUT /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/9d132112663d4c9285227f4fbccc4a94/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:45.019346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6660975-4627-404a-87ac-6c70ed7ec203 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.020122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6660975-4627-404a-87ac-6c70ed7ec203 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.020122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6660975-4627-404a-87ac-6c70ed7ec203 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.021615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 116/235] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:45.027932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb510f-bb33-4d02-b648-36a80d98f028 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.028963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb510f-bb33-4d02-b648-36a80d98f028 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.029489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb510f-bb33-4d02-b648-36a80d98f028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.054399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8cfb510f-bb33-4d02-b648-36a80d98f028 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:45.055693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8cfb510f-bb33-4d02-b648-36a80d98f028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:45.070388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:45.166299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 120/236] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:45.454773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6660975-4627-404a-87ac-6c70ed7ec203 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:45.606295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:45.610441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6660975-4627-404a-87ac-6c70ed7ec203 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:45.659388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:45.660446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.660728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.661023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.676379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f959a-991b-49b3-8039-755ff063a9f3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.677115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f959a-991b-49b3-8039-755ff063a9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.677618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f959a-991b-49b3-8039-755ff063a9f3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.679248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 121/237] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:14:45.683507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:45.685213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59332ee-257a-43c5-bdad-c883a5cd59ef None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.685701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59332ee-257a-43c5-bdad-c883a5cd59ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.686169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59332ee-257a-43c5-bdad-c883a5cd59ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:45.686704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:45.687583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:45.688373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 117/238] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:45.698427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:45.710170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a59332ee-257a-43c5-bdad-c883a5cd59ef None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:45.711714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a59332ee-257a-43c5-bdad-c883a5cd59ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:45.789564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 122/239] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:45.904909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096a47c-a848-4296-9706-fe079cdc4033 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.905636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096a47c-a848-4296-9706-fe079cdc4033 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.906155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096a47c-a848-4296-9706-fe079cdc4033 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.908145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 123/240] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:45.942006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eef144-5227-4d85-9a7a-049b04bb804e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:45.942619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eef144-5227-4d85-9a7a-049b04bb804e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:45.943162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eef144-5227-4d85-9a7a-049b04bb804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:45.979323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19942f-f068-4ee3-b1ae-2ac68fb55c7e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:45.980394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c4eef144-5227-4d85-9a7a-049b04bb804e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:45.990130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c4eef144-5227-4d85-9a7a-049b04bb804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:46.067918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:46.071498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.072125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.072601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.074436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:46.075011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:46.076690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:46.100191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 118/241] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:46.104785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 124/242] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:46.121343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:46.161283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:46.280343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771ebf1-6512-4b94-8721-0041df737ace None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:46.295353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d5dc2-6d66-428e-86de-eb70957b3265 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.295976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d5dc2-6d66-428e-86de-eb70957b3265 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.296455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d5dc2-6d66-428e-86de-eb70957b3265 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.298406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 125/243] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:46.304079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.304676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.305200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.329374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:46.332679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:46.334898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:46.341608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:46.342666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.343156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.343621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] PATH_INFO: `/v3/users/8dbd53302096499582491741359d17eb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.372030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=8dbd53302096499582491741359d17eb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:46.375586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:46.385038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 119/244] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:46 2022] GET /identity/v3/users/8dbd53302096499582491741359d17eb => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:46.385038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:46.420152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 126/245] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:46.421144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:46.556752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:46.603783 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71291078-3d37-4691-9636-3dd5c8b30bb7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:46.617619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20064f8-41fd-4afe-93b7-e8e4008f49b2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:46.661869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:46.662356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.662662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.662935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.667218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:46.672423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:46.674868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.675723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.677046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.698227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:46.708328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:46.710246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:46.711657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:46.715977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 120/246] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:46.717861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:46.759235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:46.762990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:46.845802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913cb76-730a-49e9-8105-8689ff64c952 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:46.893082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:46.893722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:46.893999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:46.894267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:46.895480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:46.896947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:46.898734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 121/247] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:46.899344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:46.910273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:46.917228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 127/248] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:46.917228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:46.964894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:47.043130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71f83d-74b2-4ebb-8722-381ee2e9787b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.048635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ae418-9771-46ca-b1c2-399620ebbb51 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.101178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.101581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.102204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.102466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.102560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.102733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.103138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.103408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:47.103756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.103843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.105446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.119895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 122/249] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:47.119895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.127676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:47.138719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:47.139706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:14:47.140498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:47.142164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 128/250] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:14:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:47.142537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.210028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:47.255018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3f2ac-ddc0-4dbd-af5f-be1113963a96 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.270590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:47.280972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906c345-dc81-474c-b0c5-4105e1dc3588 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.310315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.311261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.311743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.312189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.313976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.316491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.335043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.335888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.336265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.336640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.338817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.341500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.350097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 129/251] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:47 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:47.350097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.356062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 123/252] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:47 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:47.358026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.509989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e849-17d6-4ee6-ba75-b6806f5a8a98 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.533387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1e45-46c8-41f8-84c5-de66900fcbea None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.569877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:47.572522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.573884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.574447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.575072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] PATH_INFO: `/v3/users/62c5e70709924f08a56f4a90c9dc9ea3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.587120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=62c5e70709924f08a56f4a90c9dc9ea3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.588477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.589409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.589899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.590003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.590468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.591361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:47.591911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.597569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 130/253] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:47 2022] GET /identity/v3/users/62c5e70709924f08a56f4a90c9dc9ea3 => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:47.597569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.599536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.611206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 124/254] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:47.612423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.671759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:47.783086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1a2cf-7b72-49a4-8b00-6f0d9edc1cb0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.787988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae302af-aeb7-4926-9b81-b82e3eb0c159 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:47.838125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.839093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.839520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.839936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.841124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] RBAC: Adding query filter params (role.id=f3a873ccd0d24c608bc5c01227f253ba, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=8dbd53302096499582491741359d17eb) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:47.841597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.843777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.852154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 131/255] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:47 2022] GET /identity/v3/role_assignments?user.id=8dbd53302096499582491741359d17eb&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=f3a873ccd0d24c608bc5c01227f253ba => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:47.857044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f7995-33a2-486f-af14-5b6be2634d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:47.858196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f7995-33a2-486f-af14-5b6be2634d48 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.858415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.858833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f7995-33a2-486f-af14-5b6be2634d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.859512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f7995-33a2-486f-af14-5b6be2634d48 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.861707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f7995-33a2-486f-af14-5b6be2634d48 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:47.863845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f7995-33a2-486f-af14-5b6be2634d48 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:47.868203 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 125/256] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:47 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:47.871749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230eb5ec-669f-438f-9bea-d9c26cf1f99f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:47.872617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230eb5ec-669f-438f-9bea-d9c26cf1f99f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:47.875096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230eb5ec-669f-438f-9bea-d9c26cf1f99f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:47.875901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 126/257] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:47.877351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230eb5ec-669f-438f-9bea-d9c26cf1f99f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:47.980396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:48.049954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230eb5ec-669f-438f-9bea-d9c26cf1f99f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:48.053120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:48.067872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e4315-0681-4b48-a143-33abde620a79 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:48.114974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81494a71-c85f-4058-86e8-36bce44ebc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:48.115871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81494a71-c85f-4058-86e8-36bce44ebc44 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:48.116436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81494a71-c85f-4058-86e8-36bce44ebc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:48.116857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81494a71-c85f-4058-86e8-36bce44ebc44 None admin] PATH_INFO: `/v3/users/9d132112663d4c9285227f4fbccc4a94` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:48.118757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81494a71-c85f-4058-86e8-36bce44ebc44 None admin] RBAC: Authorizing `identity:get_user(user_id=9d132112663d4c9285227f4fbccc4a94)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:48.120537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81494a71-c85f-4058-86e8-36bce44ebc44 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:48.122686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 132/258] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:47 2022] GET /identity/v3/users/9d132112663d4c9285227f4fbccc4a94 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:48.124275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:48.124794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:48.125265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:48.128474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:48.129761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:48.130769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:48.131274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/62c5e70709924f08a56f4a90c9dc9ea3/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:48.135076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1950f1d110d94df48b26b19d33e527ed, user_id=62c5e70709924f08a56f4a90c9dc9ea3, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:48.138126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:48.152172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:48.154593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:48.165111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 127/259] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:47 2022] PUT /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/62c5e70709924f08a56f4a90c9dc9ea3/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:48.166265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:48.382939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 133/260] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:48.461165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d728d-234a-4aae-a588-c6b18ea6ade3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:48.496163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:48.533931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:48.534986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:48.535463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:48.535921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:48.692150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:48.695752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:48.915485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 128/261] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 750 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:48.921501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:49.143332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:49.160739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660c9bb-1fbc-4374-b017-084aad74cedf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:49.222324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:49.243929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:49.245440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:49.246182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:49.246884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:49.254528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:49.258638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:49.263456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 134/262] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:48 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:49.269908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:49.293723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:49.491035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cd857-336d-40c8-82be-eb88d05b48a0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:49.566505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:49.567148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:49.567420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:49.567686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:49.572161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28c35b-9c4f-4ced-bdc8-ad77c942da1f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:49.606509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:49.611587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:49.612573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:14:49.613527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:49.615740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 129/263] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:14:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:49.626028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:49.685636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:49.686266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:49.686543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:49.686811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:49.687635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=9d132112663d4c9285227f4fbccc4a94) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:49.687938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:49.689512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:49.706564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 135/264] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:49 2022] GET /identity/v3/role_assignments?user.id=9d132112663d4c9285227f4fbccc4a94&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:49.833038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:49.880432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d25bc-c0be-438a-9f24-bdcd3a821eea None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:49.951973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:49.953067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:49.953553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:49.954136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:49.955375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:49.955824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:49.957350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:49.967123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 130/265] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:14:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:49.972534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:50.041607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:50.199057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fead38f-a57b-4e85-a985-f1dc45e04d66 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:50.245076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc9108-c040-4e2b-83dc-d3a3a3d64ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:50.245727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc9108-c040-4e2b-83dc-d3a3a3d64ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:50.246001 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc9108-c040-4e2b-83dc-d3a3a3d64ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:50.246266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc9108-c040-4e2b-83dc-d3a3a3d64ab7 None admin] PATH_INFO: `/v3/users/9d132112663d4c9285227f4fbccc4a94` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:50.247469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc9108-c040-4e2b-83dc-d3a3a3d64ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=9d132112663d4c9285227f4fbccc4a94)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:50.248901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc9108-c040-4e2b-83dc-d3a3a3d64ab7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:50.250866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 136/266] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:49 2022] GET /identity/v3/users/9d132112663d4c9285227f4fbccc4a94 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:50.259968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:50.463373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0d359-cf22-4481-af90-f031f20b04c6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:50.511054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:50.511700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:50.511982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:50.512258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:50.526667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:50.624857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:50.628046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:50.685927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad030a0-a10a-4868-8466-2209cd332e63 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:50.686280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad030a0-a10a-4868-8466-2209cd332e63 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:50.686492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad030a0-a10a-4868-8466-2209cd332e63 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:50.688128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 137/267] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:50.693764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:50.694061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:50.694324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:50.720450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:50.722451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:50.847510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 138/268] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:50.878022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 131/269] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:50.883831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:50.917790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:51.016230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85add-8b49-47b3-b353-b533f40d6e09 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:51.064625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:51.065286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.065565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.065824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.066997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:51.068605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:51.071165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 139/270] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:50 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:51.076977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:51.144450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:51.230944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:51.240808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2195f-e37d-4710-90fd-9dfdec73f27e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:51.286839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:51.290111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.291428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.292524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.295621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] RBAC: Adding query filter params (role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=9d132112663d4c9285227f4fbccc4a94) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:51.296182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:51.297909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a48e7-33fe-4368-8ae5-9a805bd7e9fc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:51.312540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 132/271] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:51 2022] GET /identity/v3/role_assignments?user.id=9d132112663d4c9285227f4fbccc4a94&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:51.319927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9079-3d25-4287-887e-50b2c9aebd97 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.319927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9079-3d25-4287-887e-50b2c9aebd97 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.319927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9079-3d25-4287-887e-50b2c9aebd97 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.319927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 133/272] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:51.339615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0c892-25b7-43fa-9253-82697aab5700 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.340156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0c892-25b7-43fa-9253-82697aab5700 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.340574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0c892-25b7-43fa-9253-82697aab5700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.374327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-29b0c892-25b7-43fa-9253-82697aab5700 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:51.375852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-29b0c892-25b7-43fa-9253-82697aab5700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:51.422681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:51.440968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428a01e-a4bf-43c0-b452-bf653d1837c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:51.499190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:51.500294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.500955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.501968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.536801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:51.548139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:51.549556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:51.550594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:51.559830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 140/273] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:51.567634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:51.568038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 134/274] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:51.762651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169dadd-c535-451e-922c-b34a03bc6a99 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.762866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169dadd-c535-451e-922c-b34a03bc6a99 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.763319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169dadd-c535-451e-922c-b34a03bc6a99 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.764736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 135/275] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:51.770566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.771196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.771669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.776430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7052b-3b33-44d0-9571-04619b424070 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:51.801737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:51.803378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:51.842579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:51.844095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:51.844939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:51.845749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:51.846588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:51.848905 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:51.849786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:51.852848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:51.866462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 141/276] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:51.872113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:51.896149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:51.946687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 136/277] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:51.955672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:52.033888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bafa8-f087-4411-a4a3-cb1b26dd0556 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:52.076630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:52.077756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.078265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.078757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] PATH_INFO: `/v3/users/757b905fbbf1481fb5fcc6f299a6a5e2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.098201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=757b905fbbf1481fb5fcc6f299a6a5e2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:52.100225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:52.103618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 142/278] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:51 2022] GET /identity/v3/users/757b905fbbf1481fb5fcc6f299a6a5e2 => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:52.109840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:52.166896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace22e03-3346-4a4b-b145-a8589e984ebe None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:52.212337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:52.213032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.213387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.213668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.214976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:52.216615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:52.219163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 137/279] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:52.225070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:52.261655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:52.278658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:52.313854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f43cf-3bbd-4d3f-8b48-bd16314a5bc6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:52.362178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:52.363076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.363514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.363834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc82ff-e909-4c00-a023-0a97c5fddb99 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:52.364282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.365209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, user.id=757b905fbbf1481fb5fcc6f299a6a5e2, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:52.365701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:52.368639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:52.377732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 143/280] 10.222.0.85 () {56 vars in 1308 bytes} [Wed Nov 2 12:14:52 2022] GET /identity/v3/role_assignments?user.id=757b905fbbf1481fb5fcc6f299a6a5e2&scope.system=all&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:52.379418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:52.429388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3b990-8ba8-4b01-80e1-039d5fa527d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:52.430995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3b990-8ba8-4b01-80e1-039d5fa527d8 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.431736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3b990-8ba8-4b01-80e1-039d5fa527d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.432151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3b990-8ba8-4b01-80e1-039d5fa527d8 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.433328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3b990-8ba8-4b01-80e1-039d5fa527d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:52.437567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3b990-8ba8-4b01-80e1-039d5fa527d8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:52.490057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 138/281] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:14:52 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:14:52.494430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e995f-e053-437b-9072-715e93160430 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.494925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e995f-e053-437b-9072-715e93160430 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.495258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e995f-e053-437b-9072-715e93160430 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.498082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 139/282] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:52.503963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.504229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.504520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.528141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:52.529514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:52.578232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955dae-483d-47d5-a600-bc4c89efce01 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:52.640527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 140/283] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:52.648615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:52.650119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.650840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.651576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.656080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:52.671777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:52.680295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:52.682084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:52.683210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 144/284] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:52.688307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:52.720637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:52.888817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55129b5c-ea09-49c8-8298-c3923c6b9788 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:52.952398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:52.953400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:52.953772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:52.954170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:52.955420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:52.955935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:52.958553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:52.973972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 141/285] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:52.981429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:53.094392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:53.216848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d0e19-7e57-4608-8e28-c5eae3255e61 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:53.302355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:53.303487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:53.303979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:53.304500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] PATH_INFO: `/v3/users/8dbd53302096499582491741359d17eb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:53.314366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] RBAC: Authorizing `identity:get_user(user_id=8dbd53302096499582491741359d17eb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:53.316954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:53.320718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 145/286] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:52 2022] GET /identity/v3/users/8dbd53302096499582491741359d17eb => generated 297 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:53.321411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:53.327573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:53.385221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c44b6c-59d4-4898-b61b-07e1e4ccab9c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:53.483073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:53.530076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:53.531069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:53.531515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:53.531952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:53.564194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181892fa-0fec-4c15-aa39-6ebcdf9649df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:53.588780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:53.606491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:53.607471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:53.608518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:53.609169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 142/287] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 515 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:53.618941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:53.706312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:53.707353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:53.707826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:53.708288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:53.711760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:53.714743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:53.723884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 146/288] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:53 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:53.727552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:53.941292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a9c32-283e-4782-8376-22bef1eb2430 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:54.022645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:54.023177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:54.023612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:54.024065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:54.025115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:54.025426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:54.034944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:54.043875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:54.057987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 143/289] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:54.058264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0124b86-81be-45bc-8ece-6dec20cb9aa5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:54.064194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:54.095628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:54.112798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:54.113989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:54.114482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:54.114967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/8dbd53302096499582491741359d17eb/roles/f3a873ccd0d24c608bc5c01227f253ba` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:54.121608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] RBAC: Authorizing `identity:create_grant(project_id=1950f1d110d94df48b26b19d33e527ed, user_id=8dbd53302096499582491741359d17eb, role_id=f3a873ccd0d24c608bc5c01227f253ba)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:54.124333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:54.165649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 147/290] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:14:53 2022] PUT /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed/users/8dbd53302096499582491741359d17eb/roles/f3a873ccd0d24c608bc5c01227f253ba => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:54.360333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a18b9f-1495-4a55-8ee8-aa9cfcdd1690 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:54.440667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:54.441690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:54.442125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:54.442546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] PATH_INFO: `/v3/users/62c5e70709924f08a56f4a90c9dc9ea3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:54.445675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] RBAC: Authorizing `identity:get_user(user_id=62c5e70709924f08a56f4a90c9dc9ea3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:54.447957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:54.450609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 144/291] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:54 2022] GET /identity/v3/users/62c5e70709924f08a56f4a90c9dc9ea3 => generated 297 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:54.456917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:54.667606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368f6c-206a-4e48-bc9b-c44b7d6b8a68 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:54.742378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8330c-ef90-45ed-b231-a6dc3787a769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:54.743454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8330c-ef90-45ed-b231-a6dc3787a769 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:54.743917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8330c-ef90-45ed-b231-a6dc3787a769 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:54.744432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8330c-ef90-45ed-b231-a6dc3787a769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:54.872782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:54.894843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8330c-ef90-45ed-b231-a6dc3787a769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:54.898032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8330c-ef90-45ed-b231-a6dc3787a769 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:55.089365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 148/292] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:55.097972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:55.211259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87efaf8-110d-4606-9e21-d93e167eebc1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:55.217157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87efaf8-110d-4606-9e21-d93e167eebc1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:55.217812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87efaf8-110d-4606-9e21-d93e167eebc1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:55.220411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 149/293] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:55.227679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:55.228143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:55.228589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:55.289128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:55.291092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:55.294828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:55.377636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b053d-4caa-4aec-af27-19d5e98cfb55 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:55.455802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 150/294] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:55.481940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:55.484258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:55.485034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:55.485763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:55.488224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:55.491578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:55.495621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 145/295] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:55 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:55.505431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:55.559664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:55.725456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af08acb-7a1c-4166-b196-83a5779e6bee None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:55.788846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:55.790442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:55.790919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:55.791387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:55.792756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=62c5e70709924f08a56f4a90c9dc9ea3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:55.793299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:55.796120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c52c3-ce5c-41c0-a3b3-f1e07e0affa2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:55.808314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 151/296] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:55 2022] GET /identity/v3/role_assignments?user.id=62c5e70709924f08a56f4a90c9dc9ea3&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:55.832273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:55.959713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb721ee-e486-4440-8f10-071430ea96b6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:55.960205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb721ee-e486-4440-8f10-071430ea96b6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:55.960680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb721ee-e486-4440-8f10-071430ea96b6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:55.963617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 152/297] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:55.971208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:55.971693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:55.972181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.024276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:56.026746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:56.105846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a4238-59ed-4ab5-aada-02e87fc7ab6c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:56.182864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b28b86-3a18-4ee2-ae21-aa0dc210eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:56.183545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b28b86-3a18-4ee2-ae21-aa0dc210eb24 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.183823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b28b86-3a18-4ee2-ae21-aa0dc210eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.184093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b28b86-3a18-4ee2-ae21-aa0dc210eb24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.193545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:56.196926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b28b86-3a18-4ee2-ae21-aa0dc210eb24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f4b28b86-3a18-4ee2-ae21-aa0dc210eb24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:56.204159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:56.205148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:56.205952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 146/298] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:56.206609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081e3e6-32f3-407f-90cf-f308583bc795 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.207396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081e3e6-32f3-407f-90cf-f308583bc795 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.207396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081e3e6-32f3-407f-90cf-f308583bc795 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.213466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 147/299] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:56.216297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.216710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.217109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.219047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 153/300] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:56.220081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:56.252566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:56.261459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:56.263920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:56.351296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 148/301] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:14:56.439943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f7af3-ccf6-4324-b711-8da4bc577e3d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:56.483491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:56.484108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.484637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.484906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.485609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:56.485907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:56.487275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:56.497213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 154/302] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:56.506217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:56.527198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:56.641387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:56.671765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121518-0117-4582-8753-039b9af7bf9e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:56.700622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344f39a-f2f5-43b8-8373-e3c68a8bf59b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:56.724804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:56.725914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.726388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.727122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.728888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:56.729778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:56.731716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:56.737052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 155/303] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:14:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:56.737052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:56.745978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:56.747321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.747768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.748210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] PATH_INFO: `/v3/users/757b905fbbf1481fb5fcc6f299a6a5e2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.750086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=757b905fbbf1481fb5fcc6f299a6a5e2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:56.752594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:56.755513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 149/304] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:56 2022] GET /identity/v3/users/757b905fbbf1481fb5fcc6f299a6a5e2 => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:56.756837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:56.875533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60b141-3a5d-4368-b5b2-487938b15964 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:56.896455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a6e7b-012d-45f9-8c8e-53164af56fa0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:56.918597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:56.919681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.920097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.920701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.937221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:56.942006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:56.942626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:56.942897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:56.943164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] PATH_INFO: `/v3/groups` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:56.943816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:56.944479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:56.946444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:56.948598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:14:56.951542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:56.954510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 156/305] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:56.955667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:56.967767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 150/306] 10.222.0.85 () {60 vars in 1134 bytes} [Wed Nov 2 12:14:56 2022] POST /identity/v3/groups => generated 229 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:14:56.968046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:56.978116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:57.091461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:57.143475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a06a7a-f774-4b49-acd0-a06e1260ce3d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:57.178237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68239d34-9b5e-4af1-8708-c3d5af73f9a1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:57.190442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:57.191559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:57.192067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:57.192547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] PATH_INFO: `/v3/system/users/757b905fbbf1481fb5fcc6f299a6a5e2/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:57.195257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=757b905fbbf1481fb5fcc6f299a6a5e2, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:57.197502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:57.209358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 157/307] 10.222.0.85 () {58 vars in 1334 bytes} [Wed Nov 2 12:14:56 2022] PUT /identity/v3/system/users/757b905fbbf1481fb5fcc6f299a6a5e2/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:14:57.258051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:57.258768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:57.259040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:57.259835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:57.260530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:57.260884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:57.262465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf233-3680-4d25-ad2c-1c9ecf9d21a1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:57.275023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 151/308] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:57.281404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:57.370811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:57.411491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffccb3c-f278-4f75-a4ab-300814d91a04 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:57.426384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e1c4-7f97-4a7e-b53d-8cce447f7055 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:57.426777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e1c4-7f97-4a7e-b53d-8cce447f7055 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:57.426929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e1c4-7f97-4a7e-b53d-8cce447f7055 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:57.428375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 152/309] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:14:57.435820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:57.436094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:57.436362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:57.451619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:57.452267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:57.452545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:57.452850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] PATH_INFO: `/v3/users/1006e464cfb4493a9f8d1cbedadc8f3b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:57.470535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] RBAC: Authorizing `identity:get_user(user_id=1006e464cfb4493a9f8d1cbedadc8f3b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:57.472086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:57.474164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 158/310] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:57 2022] GET /identity/v3/users/1006e464cfb4493a9f8d1cbedadc8f3b => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:57.474320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:57.475659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:57.480550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:57.569942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 153/311] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:57.687233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d64c-d02b-4b6e-b26b-e4b686e84984 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:57.736241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:57.746207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:57.747094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:57.747598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:57.748060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:57.749478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] RBAC: Adding query filter params (role.id=f3a873ccd0d24c608bc5c01227f253ba, user.id=1006e464cfb4493a9f8d1cbedadc8f3b, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:57.750326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:57.754061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:57.763896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 159/312] 10.222.0.85 () {56 vars in 1308 bytes} [Wed Nov 2 12:14:57 2022] GET /identity/v3/role_assignments?user.id=1006e464cfb4493a9f8d1cbedadc8f3b&scope.system=all&role.id=f3a873ccd0d24c608bc5c01227f253ba => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:14:57.928564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:58.020392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:58.131820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e971f97-b5e8-4daf-b8c9-e838fddd0ee8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:58.197519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:58.198665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:58.199119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:58.199596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:58.219393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:14:58.230290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:14:58.231248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:14:58.232034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:14:58.237156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 154/313] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:14:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:14:58.243121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:58.434034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4b415-c697-4a86-bb8a-7c8f41a4b20b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:58.480696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:58.481347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:58.481620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:58.481884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:58.482552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:58.482849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:58.484240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:58.494696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 160/314] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:14:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:58.500501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:58.618703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c7d7-9cd6-4657-92e4-87c5f8373c99 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:58.629058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:58.658487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:58.659132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:58.659398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:58.659670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] PATH_INFO: `/v3/users/8dbd53302096499582491741359d17eb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:58.660909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=8dbd53302096499582491741359d17eb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:58.662427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:58.664060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 155/315] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:14:58 2022] GET /identity/v3/users/8dbd53302096499582491741359d17eb => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:58.670709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:58.744524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:58.827688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a02c71-319b-47ee-b450-02481aa2dba9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:58.872628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:58.873569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:58.873987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:58.874401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:58.951279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:58.952960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:59.013341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:59.064843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 161/316] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:14:59.073606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:59.239386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022bfb5-ce36-4e20-9188-cbe4f4616ea1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:59.283369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f377a98-3237-43a6-8377-e594532c17b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:59.283926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f377a98-3237-43a6-8377-e594532c17b5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:59.284199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f377a98-3237-43a6-8377-e594532c17b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:59.284464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f377a98-3237-43a6-8377-e594532c17b5 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:59.285819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f377a98-3237-43a6-8377-e594532c17b5 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:59.287294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f377a98-3237-43a6-8377-e594532c17b5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:59.289509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 156/317] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:14:59 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:59.300044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:14:59.433688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358307-e0d4-4560-a7c9-0e8501d2d580 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:59.434011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358307-e0d4-4560-a7c9-0e8501d2d580 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:59.434250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358307-e0d4-4560-a7c9-0e8501d2d580 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:59.436036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 157/318] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:14:59.441973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc3fbd-b8f2-49b1-b2fb-d6fb3d42b124 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:59.442235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc3fbd-b8f2-49b1-b2fb-d6fb3d42b124 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:59.442623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc3fbd-b8f2-49b1-b2fb-d6fb3d42b124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:59.450625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a77f-b11e-48b9-a55d-6d55049ea7db None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:14:59.463098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6ccc3fbd-b8f2-49b1-b2fb-d6fb3d42b124 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:59.465122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6ccc3fbd-b8f2-49b1-b2fb-d6fb3d42b124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:59.491979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:14:59.492412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:59.492693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:59.492969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:59.493882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] RBAC: Adding query filter params (role.id=f3a873ccd0d24c608bc5c01227f253ba, scope.project.id=1950f1d110d94df48b26b19d33e527ed, user.id=8dbd53302096499582491741359d17eb) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:14:59.494184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:14:59.495657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c53d08-a934-453b-ba6e-930897ac20db None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:14:59.503603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 162/319] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:14:59 2022] GET /identity/v3/role_assignments?user.id=8dbd53302096499582491741359d17eb&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=f3a873ccd0d24c608bc5c01227f253ba => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:14:59.549954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:14:59.587955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 158/320] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:14:59.854750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e168-8ba4-4724-a34a-76fbb94e60d4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:59.855241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e168-8ba4-4724-a34a-76fbb94e60d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:59.855674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e168-8ba4-4724-a34a-76fbb94e60d4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:59.858042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 163/321] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:14:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:14:59.864220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:14:59.864697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:14:59.864770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:14:59.884307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:14:59.886141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:14:59.966136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 159/322] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:00.063437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5e168-8ba4-4724-a34a-76fbb94e60d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:00.100063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:00.201525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5e168-8ba4-4724-a34a-76fbb94e60d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:00.238462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:00.240076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:00.240723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.241037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.241376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.242586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:00.244153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:00.246531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 164/323] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:00.254748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:00.372807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:00.378813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef92307-4257-4af9-a0f4-1e1a29c84a39 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:00.381587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:00.400533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337160b7-6621-437c-a5e5-0355959d83c3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:00.429335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95460566-046c-4873-b7d5-4570ab3d78a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:00.429980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95460566-046c-4873-b7d5-4570ab3d78a8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.430253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95460566-046c-4873-b7d5-4570ab3d78a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.430517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95460566-046c-4873-b7d5-4570ab3d78a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.442264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95460566-046c-4873-b7d5-4570ab3d78a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-95460566-046c-4873-b7d5-4570ab3d78a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:00.447194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:00.448216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:00.449062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 160/324] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:00.449746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc10c9-9a45-40eb-975f-531ddde5b901 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.449970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc10c9-9a45-40eb-975f-531ddde5b901 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.450653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc10c9-9a45-40eb-975f-531ddde5b901 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.452054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 161/325] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:00.454131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5f5c8-40c3-4210-8bb3-b25a7ef6f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:00.454492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc10c9-9a45-40eb-975f-531ddde5b901 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:00.455304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5f5c8-40c3-4210-8bb3-b25a7ef6f847 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.455785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5f5c8-40c3-4210-8bb3-b25a7ef6f847 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.456271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5f5c8-40c3-4210-8bb3-b25a7ef6f847 None admin] PATH_INFO: `/v3/groups` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.457456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5f5c8-40c3-4210-8bb3-b25a7ef6f847 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:00.459449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5f5c8-40c3-4210-8bb3-b25a7ef6f847 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:00.482424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 165/326] 10.222.0.85 () {60 vars in 1134 bytes} [Wed Nov 2 12:15:00 2022] POST /identity/v3/groups => generated 226 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:15:00.483696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.483978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.484343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.515362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:00.516943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:00.595382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc10c9-9a45-40eb-975f-531ddde5b901 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:00.634666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 166/327] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:00.655814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:00.656537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.656941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.657320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.658167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:00.658580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:00.660089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:00.672691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 162/328] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:00.682843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:00.784326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:00.828462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9584f-212d-49e6-8e28-4b1bd53bc9b5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:00.874957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:00.875728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:00.875855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:00.876113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] PATH_INFO: `/v3/users/757b905fbbf1481fb5fcc6f299a6a5e2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:00.877328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] RBAC: Authorizing `identity:get_user(user_id=757b905fbbf1481fb5fcc6f299a6a5e2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:00.878793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:00.880795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 167/329] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:15:00 2022] GET /identity/v3/users/757b905fbbf1481fb5fcc6f299a6a5e2 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:00.889696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:00.924733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:00.956748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:01.044525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f63e22-3002-41a7-bebb-e730cadcfde3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:01.092859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:01.093687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:01.094013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:01.094298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:01.095029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] RBAC: Adding query filter params (role.id=5ccb31a782a44a4aad62b2f6bba3d582, user.id=757b905fbbf1481fb5fcc6f299a6a5e2, scope.system=all) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:01.095332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:01.097483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:01.108982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7ea2-1b26-40a5-bbdc-7405b3ba6120 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:01.119600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 163/330] 10.222.0.85 () {56 vars in 1308 bytes} [Wed Nov 2 12:15:00 2022] GET /identity/v3/role_assignments?user.id=757b905fbbf1481fb5fcc6f299a6a5e2&scope.system=all&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:01.174411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:01.175495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:01.175973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:01.176438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:01.194514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:01.201623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:01.202591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:15:01.203455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:01.206173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 168/331] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:01.212606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:01.344507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:01.372557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253e694-f30c-4e53-a9e8-31a30c937ca5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:01.421085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:01.422166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:01.422679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:01.423170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:01.424400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:01.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:01.427138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:01.442510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 164/332] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:01.449401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:01.457214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:01.645654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a705ed-fedc-4d03-9a69-cd758df7031c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:01.698206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ca993-8204-4a43-a5f9-17e2e0b5d354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:01.698919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ca993-8204-4a43-a5f9-17e2e0b5d354 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:01.699195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ca993-8204-4a43-a5f9-17e2e0b5d354 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:01.699470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ca993-8204-4a43-a5f9-17e2e0b5d354 None admin] PATH_INFO: `/v3/users/1006e464cfb4493a9f8d1cbedadc8f3b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:01.701195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ca993-8204-4a43-a5f9-17e2e0b5d354 None admin] RBAC: Authorizing `identity:get_user(user_id=1006e464cfb4493a9f8d1cbedadc8f3b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:01.702716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ca993-8204-4a43-a5f9-17e2e0b5d354 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:01.704667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 169/333] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:15:01 2022] GET /identity/v3/users/1006e464cfb4493a9f8d1cbedadc8f3b => generated 293 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:01.711166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:01.861145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0337-c51e-4091-89fc-ae2a8727cb05 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:01.902782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9209f3-0686-463a-b6a7-f50bfad3be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:01.903390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9209f3-0686-463a-b6a7-f50bfad3be10 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:01.903628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9209f3-0686-463a-b6a7-f50bfad3be10 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:01.903943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9209f3-0686-463a-b6a7-f50bfad3be10 None admin] PATH_INFO: `/v3/system/users/1006e464cfb4493a9f8d1cbedadc8f3b/roles/f3a873ccd0d24c608bc5c01227f253ba` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:01.905452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9209f3-0686-463a-b6a7-f50bfad3be10 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1006e464cfb4493a9f8d1cbedadc8f3b, role_id=f3a873ccd0d24c608bc5c01227f253ba)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:01.906853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9209f3-0686-463a-b6a7-f50bfad3be10 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:01.921154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 165/334] 10.222.0.85 () {58 vars in 1334 bytes} [Wed Nov 2 12:15:01 2022] PUT /identity/v3/system/users/1006e464cfb4493a9f8d1cbedadc8f3b/roles/f3a873ccd0d24c608bc5c01227f253ba => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:15:04.092466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f827a73-e8a4-4711-a110-bddc2e96d45e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:04.093311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f827a73-e8a4-4711-a110-bddc2e96d45e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:04.093440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f827a73-e8a4-4711-a110-bddc2e96d45e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:04.096084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 170/335] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:04.102331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507f3a-8596-4ae3-a46d-b3ee3b8128d3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:04.103045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507f3a-8596-4ae3-a46d-b3ee3b8128d3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:04.103212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507f3a-8596-4ae3-a46d-b3ee3b8128d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:04.141134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-44507f3a-8596-4ae3-a46d-b3ee3b8128d3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:04.143271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-44507f3a-8596-4ae3-a46d-b3ee3b8128d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:04.148235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:04.300800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 166/336] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:04.806780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f827a73-e8a4-4711-a110-bddc2e96d45e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:04.966454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:05.033738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f827a73-e8a4-4711-a110-bddc2e96d45e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:05.110149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84942c39-947c-4921-b463-b957065995c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:05.111096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84942c39-947c-4921-b463-b957065995c2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.111502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84942c39-947c-4921-b463-b957065995c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.111939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84942c39-947c-4921-b463-b957065995c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.143276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84942c39-947c-4921-b463-b957065995c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:05.147827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191165e0-4bdd-480b-9844-ff98ede9cf8c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.153217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191165e0-4bdd-480b-9844-ff98ede9cf8c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.154101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191165e0-4bdd-480b-9844-ff98ede9cf8c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.157335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 167/337] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-84942c39-947c-4921-b463-b957065995c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:05.157495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:05.159114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:05.160674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 171/338] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:05.167328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0c597-eb03-4744-bfbe-225737183c5f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.167328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0c597-eb03-4744-bfbe-225737183c5f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.167328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0c597-eb03-4744-bfbe-225737183c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.181711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191165e0-4bdd-480b-9844-ff98ede9cf8c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:05.226936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8dc0c597-eb03-4744-bfbe-225737183c5f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:05.229483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8dc0c597-eb03-4744-bfbe-225737183c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:05.451630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 172/339] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:05.525049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191165e0-4bdd-480b-9844-ff98ede9cf8c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:05.563538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae93a-8922-4e0f-90bd-44c78357408c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.564491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae93a-8922-4e0f-90bd-44c78357408c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.565359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae93a-8922-4e0f-90bd-44c78357408c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.574457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 173/340] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:05 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:05.595011 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f21fe-fddc-444a-8ab4-489e63c1f5ac None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.595879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f21fe-fddc-444a-8ab4-489e63c1f5ac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.596578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f21fe-fddc-444a-8ab4-489e63c1f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.646112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:05.646982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.647407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.647825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.653030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:05.656142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:05.665030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-195f21fe-fddc-444a-8ab4-489e63c1f5ac None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:05.665030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-195f21fe-fddc-444a-8ab4-489e63c1f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:05.667133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:05.681624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:05.697211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 168/341] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 517 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:05.703335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:05.793709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:05.808185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 174/342] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:05.810567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb97286-e3e2-455b-a1e8-c4d3934e9f35 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.811091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb97286-e3e2-455b-a1e8-c4d3934e9f35 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.811888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb97286-e3e2-455b-a1e8-c4d3934e9f35 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.813790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 175/343] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:05.833332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:05.834279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:05.834939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:05.922583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:05.925045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:06.045123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf56d0-a676-4b6d-a97e-e90a7b636f4e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:06.186525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:06.188055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:06.188616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:06.189111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:06.241028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 176/344] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:06.319107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:06.325766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:06.327687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:06.398699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:06.575966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac199bff-00f8-40e9-839e-b115840ecb85 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:06.605271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:06.621719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:06.622889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:06.623366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:06.623837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:06.637649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:06.643171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:06.644105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:06.644894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:06.647510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 177/345] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:06.654630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:06.696810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 169/346] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 994 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:15:06.698408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:06.795484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3038d-510b-47b6-bdf6-5c9e1fe5e492 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:06.842597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81968750-c5de-4317-9719-76057668adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:06.843654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81968750-c5de-4317-9719-76057668adeb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:06.844139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81968750-c5de-4317-9719-76057668adeb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:06.844631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81968750-c5de-4317-9719-76057668adeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:06.859213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81968750-c5de-4317-9719-76057668adeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-81968750-c5de-4317-9719-76057668adeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:06.865706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:06.866697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:06.867596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:06.875599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 178/347] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:06.877038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81968750-c5de-4317-9719-76057668adeb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:06.932243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:07.022044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81968750-c5de-4317-9719-76057668adeb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.032365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:07.038375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad107c-1a41-4364-ae86-4676fa92adaf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.087424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.088179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.088526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.088867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.090252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:07.091721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:07.093570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 179/348] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:06 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:07.094373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:07.111931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.112938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.113463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.113934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.115417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:07.115937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:07.118514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:07.135427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 170/349] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:07.136899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:07.260668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57e63b-4fd5-44f7-9936-854082331984 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.284156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:07.335026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.336034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.336749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.338450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.339455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:07.343044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:07.343044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76b35c-bc88-4273-896e-f8f5f7071eb1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:07.352892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f37f1-9f04-45f0-9f50-abd66b858b36 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.358790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 180/350] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:07.361809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857a4da-7dfe-4a00-b62f-3920eef6e617 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.362329 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857a4da-7dfe-4a00-b62f-3920eef6e617 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.362868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857a4da-7dfe-4a00-b62f-3920eef6e617 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.365715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 181/351] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:07.373372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857a4da-7dfe-4a00-b62f-3920eef6e617 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:07.426139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.426804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.427090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.427592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.448120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:07.460214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:07.461943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 171/352] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:07.465335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:07.511194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:07.526833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857a4da-7dfe-4a00-b62f-3920eef6e617 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.575249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.575901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.576221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.576498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.585926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:07.587428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:07.590972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 182/353] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:07 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:07.592392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:07.640547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d25427-281e-42df-9966-29dc433349c2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.650405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:07.702276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990f9d-17bf-4a22-acb6-ab60097ce558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.703564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990f9d-17bf-4a22-acb6-ab60097ce558 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.704249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990f9d-17bf-4a22-acb6-ab60097ce558 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.704672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990f9d-17bf-4a22-acb6-ab60097ce558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.775141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc72e0c-1408-413f-ae8e-e7dfade82781 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:07.831617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990f9d-17bf-4a22-acb6-ab60097ce558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:07.834477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990f9d-17bf-4a22-acb6-ab60097ce558 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:07.834987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068abac6-316e-419c-8f87-84eef70c4fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:07.836131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068abac6-316e-419c-8f87-84eef70c4fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:07.836758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068abac6-316e-419c-8f87-84eef70c4fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:07.837443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068abac6-316e-419c-8f87-84eef70c4fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:07.883086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:07.949306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068abac6-316e-419c-8f87-84eef70c4fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:07.950805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068abac6-316e-419c-8f87-84eef70c4fd8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:08.004272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:08.016728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 172/354] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:08.018915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.019396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.019864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.049340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:08.051933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:08.107253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 183/355] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:08.109185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29da1f-46d9-4a6d-bc7a-c6d8be317800 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.109757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29da1f-46d9-4a6d-bc7a-c6d8be317800 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.111209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29da1f-46d9-4a6d-bc7a-c6d8be317800 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.112822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 184/356] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:08.114252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29da1f-46d9-4a6d-bc7a-c6d8be317800 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:08.168426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 173/357] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:08.169713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:08.268831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:08.305918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29da1f-46d9-4a6d-bc7a-c6d8be317800 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:08.338878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e8865-1e48-4e60-8596-0ddadafe7d66 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:08.377374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:08.378396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.378845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.379260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.380410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:08.380936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:08.382738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:08.383424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.383764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.383999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.384676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=86c6007706a043bfa43e23a20cdbe653) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:08.384982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:08.385154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:08.386457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:08.389972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:08.396777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 174/358] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:08 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:08.398979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:08.415671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 185/359] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:08.422514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:08.602829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d8e7-b008-48ba-a141-4081a4f2dae4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:08.638228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:08.649903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723565d-b4e9-4f65-b1e3-86cee6633271 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:08.676580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bd440-4a3b-422a-8cc1-98d9e5a9f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:08.678520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bd440-4a3b-422a-8cc1-98d9e5a9f74e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.679024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bd440-4a3b-422a-8cc1-98d9e5a9f74e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.679870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bd440-4a3b-422a-8cc1-98d9e5a9f74e None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.682965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bd440-4a3b-422a-8cc1-98d9e5a9f74e None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:08.687572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bd440-4a3b-422a-8cc1-98d9e5a9f74e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:08.692420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 175/360] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:08 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:08.694355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.694844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.695306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.702026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:08.702683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.703018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.703293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:08.705476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:08.707023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:08.709179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 186/361] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:08 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:08.710019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:08.732319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:08.734141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:08.797035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:08.847176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b69f67-a495-4577-9433-915bb9657a13 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:08.873714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 176/362] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:08.874514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:08.922820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:08.924508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:08.925455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:08.926227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.077729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:09.080696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f303-ffbb-4642-9e58-9325c67b3b18 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:09.081322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:09.105541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:09.136213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:09.136819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:09.137138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:09.137471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.139871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:09.141403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:09.143190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 177/363] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:08 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:09.144142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.218087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:09.288408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 187/364] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:09.288408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.309339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3633e0f-3706-4577-b8e5-e2225ff54381 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:09.349637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:09.350263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:09.350537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:09.350806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.363724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:09.369862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:09.370843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 178/365] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:09.372620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.442909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:09.443977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a4fe5-ff34-44f9-a9e0-41c5b43ac81b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:09.506275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:09.507337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:09.507825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:09.508311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.510427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:09.513163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:09.515872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 188/366] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:09.517378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.534701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026bd6d-6634-47c7-b436-8c12f0a8a35e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:09.570929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:09.604570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:09.605653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:09.606152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:09.606641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.608031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=86c6007706a043bfa43e23a20cdbe653) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:09.609143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:09.611895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:09.633031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 179/367] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:09.633031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.691090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a3547-c657-478f-9e33-438bfb71b445 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:09.745962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:09.746516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:09.746818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:09.747072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.762917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:09.768164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:09.769215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:09.770705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 189/368] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:09.771374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.819794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93b808-07b8-4c7d-bb4d-7470643646dc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:09.830745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:09.869908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:09.872171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:09.872538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:09.872797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:09.874254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:09.875890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:09.878001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 180/369] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:09.878876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:09.920142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:09.943374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407031-3931-4726-b6b8-8f7f54105d61 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.012087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:10.013401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:10.013938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:10.014969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:10.016167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:10.016661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:10.019129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:10.035380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 190/370] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:10.045034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:10.068951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b5454-d823-4dba-b965-26011cd03261 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.139312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:10.140454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:10.140934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:10.141353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:10.143009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=1950f1d110d94df48b26b19d33e527ed) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:10.143555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:10.145950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:10.157254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 181/371] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:09 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:10.158548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:10.228033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeb263-b01e-419d-bd3e-9d3cd6fa0189 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.243601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:10.332203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:10.342529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:10.343283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:10.343754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:10.344728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:10.345307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:10.347026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:10.367911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 191/372] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:10.373332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:10.403936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a1b71-22d6-4f66-8c26-3dc918726b20 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.423706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:10.492428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:10.493542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:10.494012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:10.494531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:10.502462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:10.505478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:10.508771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 182/373] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:10 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:10.510393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:10.600547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4eafe-9ead-492c-86ef-86b65c09ba4e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.701337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:10.702777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:10.703643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:10.704241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] PATH_INFO: `/v3/users/1006e464cfb4493a9f8d1cbedadc8f3b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:10.705877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] RBAC: Authorizing `identity:get_user(user_id=1006e464cfb4493a9f8d1cbedadc8f3b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:10.707659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:10.713951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 192/374] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:15:10 2022] GET /identity/v3/users/1006e464cfb4493a9f8d1cbedadc8f3b => generated 293 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:10.715026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:10.783812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eee86-2941-4f7a-9390-658d31e73b3c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.796639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:10.857714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:10.859237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:10.859960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:10.861309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:10.905553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:10.934890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d8596-9100-4a12-a5b6-a67d18e61c36 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:10.973429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:10.975314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:11.075641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:11.076695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:11.077240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:11.077773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:11.079183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=1950f1d110d94df48b26b19d33e527ed) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:11.079728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:11.082570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:11.090831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 183/375] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:11.092046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:11.101650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:11.108362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 193/376] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:10 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 260 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:11.110394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:11.251262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42e4f4-1dc2-4eaa-85dc-ab626b4822e4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:11.344256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:11.345306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:11.345777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:11.346237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:11.347165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] RBAC: Adding query filter params (role.id=f3a873ccd0d24c608bc5c01227f253ba, user.id=1006e464cfb4493a9f8d1cbedadc8f3b, scope.system=all) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:11.347686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:11.349375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:11.384898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 184/377] 10.222.0.85 () {56 vars in 1308 bytes} [Wed Nov 2 12:15:11 2022] GET /identity/v3/role_assignments?user.id=1006e464cfb4493a9f8d1cbedadc8f3b&scope.system=all&role.id=f3a873ccd0d24c608bc5c01227f253ba => generated 510 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:11.431900 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fb06-27e6-4c82-a5be-dbd6390dba05 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:11.488799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:11.489483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:11.493507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:11.493783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] PATH_INFO: `/v3/projects/bbb8bdae345348eca7be88e01d7954f8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:11.495159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] RBAC: Authorizing `identity:get_project(project_id=bbb8bdae345348eca7be88e01d7954f8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:11.496667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:11.498959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 194/378] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:11 2022] GET /identity/v3/projects/bbb8bdae345348eca7be88e01d7954f8 => generated 344 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:11.507325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:11.619155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:11.672775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034c992-2380-45f9-8ce7-8c879a43dd5c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:11.715643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe60ae-11d8-44ac-9198-f1c4d8683eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:11.716288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe60ae-11d8-44ac-9198-f1c4d8683eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:11.716666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe60ae-11d8-44ac-9198-f1c4d8683eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:11.716908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe60ae-11d8-44ac-9198-f1c4d8683eb0 None admin] PATH_INFO: `/v3/groups/a8c31117c90a4a239ea8ae11f30e8fe2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:11.729982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe60ae-11d8-44ac-9198-f1c4d8683eb0 None admin] RBAC: Authorizing `identity:get_group(group_id=a8c31117c90a4a239ea8ae11f30e8fe2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:11.731457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe60ae-11d8-44ac-9198-f1c4d8683eb0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:11.733526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 185/379] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:11 2022] GET /identity/v3/groups/a8c31117c90a4a239ea8ae11f30e8fe2 => generated 229 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:11.742471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:11.800119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:11.802125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b347a-d818-4dd7-8c1f-5e9a7e299813 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:11.802347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b347a-d818-4dd7-8c1f-5e9a7e299813 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:11.802622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b347a-d818-4dd7-8c1f-5e9a7e299813 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:11.803986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 186/380] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:15:11.823880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:11.824352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:11.824838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:11.860945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:11.862919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:11.989998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7040c-3931-4f42-a6f2-9e027842c777 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:12.042637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 187/381] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:12.068362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:12.069405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:12.069870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:12.070333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:12.071629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] RBAC: Adding query filter params (group.id=a8c31117c90a4a239ea8ae11f30e8fe2, role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=bbb8bdae345348eca7be88e01d7954f8) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:12.072156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:12.074704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:12.094237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 195/382] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:11 2022] GET /identity/v3/role_assignments?group.id=a8c31117c90a4a239ea8ae11f30e8fe2&scope.project.id=bbb8bdae345348eca7be88e01d7954f8&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 260 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:12.352716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:12.432972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:12.495030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c01f6-40dc-49ce-94d9-c422eadf59d1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:12.539387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:12.540138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:12.540295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:12.540553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:12.558114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:12.563428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:12.564396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:12.565230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:12.569206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 188/383] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:12.571086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:12.684089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:12.704934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8e972-2c8f-4232-86c4-d907f2061efa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:12.747791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:12.748752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:12.748883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:12.749261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:12.749820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:12.750125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:12.751522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:12.763594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 196/384] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:12.769800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:12.899610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809a3df-3858-48e1-add6-cb407967cfa8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:12.925705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:12.956047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:12.957160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:12.957815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:12.958289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:12.963010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:12.965643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:12.971280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 189/385] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:12 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:12.979161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:13.126014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff58ef-90b8-4bed-a3e0-b36fcd71a210 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:13.166240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9c81c-2725-4e7f-b4c4-1aa64e34b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:13.166948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9c81c-2725-4e7f-b4c4-1aa64e34b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:13.167226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9c81c-2725-4e7f-b4c4-1aa64e34b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:13.167491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9c81c-2725-4e7f-b4c4-1aa64e34b1e6 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:13.171088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9c81c-2725-4e7f-b4c4-1aa64e34b1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:13.173330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9c81c-2725-4e7f-b4c4-1aa64e34b1e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:13.175255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 197/386] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:12 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:13.182041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:13.195874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604a3e3-c3b3-4386-b16d-aa701fe5d4d6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:13.196201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604a3e3-c3b3-4386-b16d-aa701fe5d4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:13.196421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604a3e3-c3b3-4386-b16d-aa701fe5d4d6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:13.198171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 198/387] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:15:13.209019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6655-bc81-4e7b-a5d4-3fb41b1f8add None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:13.209303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6655-bc81-4e7b-a5d4-3fb41b1f8add None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:13.209501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6655-bc81-4e7b-a5d4-3fb41b1f8add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:13.247831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1fbf6655-bc81-4e7b-a5d4-3fb41b1f8add None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:13.249186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fbf6655-bc81-4e7b-a5d4-3fb41b1f8add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:13.263333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:13.380897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380aa3-bcab-4a42-bfa4-646ef21ae27d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:13.392941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 199/388] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:13.439702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a54b-89c8-456f-8593-f63c7569ee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:13.440489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a54b-89c8-456f-8593-f63c7569ee57 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:13.440759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a54b-89c8-456f-8593-f63c7569ee57 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:13.441045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a54b-89c8-456f-8593-f63c7569ee57 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653/groups/29c2dc8308944058901de5ade635d264/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:13.444734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4a54b-89c8-456f-8593-f63c7569ee57 None admin] RBAC: Authorizing `identity:create_grant(project_id=86c6007706a043bfa43e23a20cdbe653, group_id=29c2dc8308944058901de5ade635d264, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:13.446354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4a54b-89c8-456f-8593-f63c7569ee57 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:13.470503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 190/389] 10.222.0.85 () {58 vars in 1442 bytes} [Wed Nov 2 12:15:13 2022] PUT /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653/groups/29c2dc8308944058901de5ade635d264/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:15:13.702529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c435c-e453-4634-afca-0d6b2254912d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:13.702783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c435c-e453-4634-afca-0d6b2254912d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:13.703216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c435c-e453-4634-afca-0d6b2254912d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:13.704956 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 200/390] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:13.711550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:13.712320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:13.712320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:13.736856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c435c-e453-4634-afca-0d6b2254912d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:13.759495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:13.760737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:13.851384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:13.888267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 191/391] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:13.955801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c435c-e453-4634-afca-0d6b2254912d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.011738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:14.012310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:14.012591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:14.012864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:14.031129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:14.037208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:14.038388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:14.039485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 201/392] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:14.049262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:14.194709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:14.198413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:14.210966 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29451078-6604-4ed5-bdb3-dde4b86b987c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.315249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:14.316221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:14.316692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:14.317196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:14.318348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:14.318883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:14.321367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:14.347576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 192/393] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:14.357170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:14.363826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa26f4-7a9b-4cb1-920d-0f0d1edc908d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.437780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:14.443991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:14.445026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:14.445521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:14.445982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:14.460714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:14.466995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:14.468170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:14.469018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:14.477387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 202/394] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:14.483659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:14.568778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7ea9b-d27e-4736-9116-3f32cac50718 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.618303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:14.619892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:14.620660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:14.621485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:14.623891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:14.627006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:14.630786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 193/395] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:14 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:14.637571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:14.643544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:14.696038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d959b9-145c-4c23-9ad7-ff0c5ad99a99 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.744436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:14.745497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:14.745977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:14.746478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:14.747385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:14.747902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:14.749652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:14.767838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 203/396] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:14.770064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:14.942668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087fe73-e8df-4679-868a-1aff603a344f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.970623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ccdfe-1b94-4e8b-8b25-781389484cc4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:14.983314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:14.986738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:15.014926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:15.015570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.015848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.016118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.017950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:15.019691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:15.021981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 204/397] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:14 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:15.029673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:15.053906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:15.054556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.054831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.055100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.056421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:15.057920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:15.059951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 194/398] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:14 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:15.067212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:15.204306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253211ea-98c5-4c08-a570-3104ab481b1f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:15.226694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b001228-579d-43f0-a9c1-6eb755856131 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:15.250003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ead52-6f88-4386-b778-31ab904138b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:15.250732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ead52-6f88-4386-b778-31ab904138b2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.251564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ead52-6f88-4386-b778-31ab904138b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.251870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ead52-6f88-4386-b778-31ab904138b2 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.253157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ead52-6f88-4386-b778-31ab904138b2 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:15.254642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ead52-6f88-4386-b778-31ab904138b2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:15.256704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 205/399] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:15 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:15.258016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a8ae0-d68c-4d53-bba9-afc7bc548fba None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.258301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a8ae0-d68c-4d53-bba9-afc7bc548fba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.258572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a8ae0-d68c-4d53-bba9-afc7bc548fba None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.260520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 206/400] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:15.270527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff446774-ff84-4eb3-929e-2daefb7a0631 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.271035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff446774-ff84-4eb3-929e-2daefb7a0631 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.271487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff446774-ff84-4eb3-929e-2daefb7a0631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.290772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff446774-ff84-4eb3-929e-2daefb7a0631 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:15.292400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff446774-ff84-4eb3-929e-2daefb7a0631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:15.293786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:15.295219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.295755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.296311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed/groups/29c2dc8308944058901de5ade635d264/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.300566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1950f1d110d94df48b26b19d33e527ed, group_id=29c2dc8308944058901de5ade635d264, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:15.303283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:15.332651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 195/401] 10.222.0.85 () {58 vars in 1442 bytes} [Wed Nov 2 12:15:15 2022] PUT /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed/groups/29c2dc8308944058901de5ade635d264/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:15:15.334601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:15.455714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:15.457325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:15.511567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 207/402] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:15.581457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a412211-310f-4931-a628-68f10f5d98d9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:15.628776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:15.629491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.629778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.630478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653/groups/29c2dc8308944058901de5ade635d264/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.632795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] RBAC: Authorizing `identity:create_grant(project_id=86c6007706a043bfa43e23a20cdbe653, group_id=29c2dc8308944058901de5ade635d264, role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:15.634290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:15.646269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0de0c-8560-4a8c-addd-8b73ef4c2338 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.646580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0de0c-8560-4a8c-addd-8b73ef4c2338 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.646819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0de0c-8560-4a8c-addd-8b73ef4c2338 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.652206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 208/403] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:15.655202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:15.655793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:15.656277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:15.656380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 196/404] 10.222.0.85 () {58 vars in 1442 bytes} [Wed Nov 2 12:15:15 2022] PUT /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653/groups/29c2dc8308944058901de5ade635d264/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:15.702680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:15.704892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:15.826748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:15.841136 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 209/405] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:16.035857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:16.046738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1a090-702f-41fd-8cb4-a574543969dc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:16.087620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:16.088167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.088438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.088701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.102835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:16.108324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:16.109935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:16.112271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 197/406] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:16.116425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:16.150014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:16.302449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e37c6b-1e88-4392-9308-08c6eaeea4f1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:16.377285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:16.400613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4628a-9453-4294-9b7d-8a827b24dfb4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:16.441022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91918f-b90b-4e08-93af-d236b461eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:16.441573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:16.441744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91918f-b90b-4e08-93af-d236b461eca9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.441968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91918f-b90b-4e08-93af-d236b461eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.442242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91918f-b90b-4e08-93af-d236b461eca9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.443536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.444440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.445392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.446883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:16.447692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:16.450979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3dda3-86fa-43bc-84f2-a4c2c59d0358 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:16.453995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91918f-b90b-4e08-93af-d236b461eca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-cd91918f-b90b-4e08-93af-d236b461eca9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:16.460540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:16.462216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 198/407] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:16.463752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7041d-c5ad-4bf2-8c3c-6b42fb63b605 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.465157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7041d-c5ad-4bf2-8c3c-6b42fb63b605 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.465157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7041d-c5ad-4bf2-8c3c-6b42fb63b605 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.470921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 199/408] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:16.477285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7041d-c5ad-4bf2-8c3c-6b42fb63b605 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:16.477491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 210/409] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:16.483418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.484271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.485038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.503348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:16.518731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:16.520944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:16.615703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7041d-c5ad-4bf2-8c3c-6b42fb63b605 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:16.641155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 211/410] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:16.641155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:16.727574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:16.733190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.734223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.734967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.737149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:16.737954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:16.740989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:16.759095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af7b8a-c49d-4dc1-bdaf-8844a52f9c70 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:16.774590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 200/411] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:16.774873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:16.781079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:16.815647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:16.818160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:16.819340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:16.821413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:16.823865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:16.829919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:16.839344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 212/412] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:16 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:16.843161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.016961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:17.084424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86ed3-f5a0-42ed-8937-8afd8526b9e9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.109713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62233b-0270-4558-a345-6737e2535591 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.152696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.153772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.153772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.154061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.155057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:17.155394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.156795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:17.156973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.158290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.159057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] PATH_INFO: `/v3/groups/a8c31117c90a4a239ea8ae11f30e8fe2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.159390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 213/413] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:16 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:17.160183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.162159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] RBAC: Authorizing `identity:get_group(group_id=a8c31117c90a4a239ea8ae11f30e8fe2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:17.165347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:17.169105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 201/414] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:16 2022] GET /identity/v3/groups/a8c31117c90a4a239ea8ae11f30e8fe2 => generated 229 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:17.170753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.222282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:17.285521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1b4b-7eb4-466f-9265-af639a78eb98 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.335681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471cbd-69fb-4f0a-b805-f78a1dd04b45 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.335934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.336603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.336876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.337181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.354230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:17.360599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:17.362981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 214/415] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:17.364597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.377575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.378315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.378595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.378841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed/groups/29c2dc8308944058901de5ade635d264/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.381344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] RBAC: Authorizing `identity:create_grant(project_id=1950f1d110d94df48b26b19d33e527ed, group_id=29c2dc8308944058901de5ade635d264, role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:17.382839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:17.400535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 202/416] 10.222.0.85 () {58 vars in 1442 bytes} [Wed Nov 2 12:15:17 2022] PUT /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed/groups/29c2dc8308944058901de5ade635d264/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:17.401564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.405741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:17.500906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e495f6-b573-4dab-b157-db70bde8d02a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.513490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:17.542581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.543160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.543436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.543780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] PATH_INFO: `/v3/projects/bbb8bdae345348eca7be88e01d7954f8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.546317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] RBAC: Authorizing `identity:get_project(project_id=bbb8bdae345348eca7be88e01d7954f8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:17.547827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:17.550012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 215/417] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:17 2022] GET /identity/v3/projects/bbb8bdae345348eca7be88e01d7954f8 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:17.557149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.604793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80939a-1172-4636-83e3-bcae8d882fdb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.697698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b63d0-84e4-4ff4-b1ba-69d5936d6644 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.718771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.719390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.719665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.719939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.721171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:17.721473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:17.723235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:17.739459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 203/418] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:17.741875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:17.756590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.758030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.758819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.759425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] PATH_INFO: `/v3/projects/bbb8bdae345348eca7be88e01d7954f8/groups/a8c31117c90a4a239ea8ae11f30e8fe2/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:17.761757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] RBAC: Authorizing `identity:create_grant(project_id=bbb8bdae345348eca7be88e01d7954f8, group_id=a8c31117c90a4a239ea8ae11f30e8fe2, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:17.763419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:17.781746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 216/419] 10.222.0.85 () {58 vars in 1442 bytes} [Wed Nov 2 12:15:17 2022] PUT /identity/v3/projects/bbb8bdae345348eca7be88e01d7954f8/groups/a8c31117c90a4a239ea8ae11f30e8fe2/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:15:17.840713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:17.897205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b30a-5c67-4390-b248-3c068285e7a4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:17.943251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9e903-4d5b-4a5c-928e-299ae355aff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:17.943895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9e903-4d5b-4a5c-928e-299ae355aff7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:17.944181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9e903-4d5b-4a5c-928e-299ae355aff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:17.944455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9e903-4d5b-4a5c-928e-299ae355aff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.025428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9e903-4d5b-4a5c-928e-299ae355aff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:18.026933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9e903-4d5b-4a5c-928e-299ae355aff7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:18.106927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:18.157198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 204/420] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:18.163493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:18.233119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:18.334061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab694c-88f8-450f-b08a-1a16a1469d7a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:18.371195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2213c-60dd-46f8-aa61-91515b181e27 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.371490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2213c-60dd-46f8-aa61-91515b181e27 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.371668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2213c-60dd-46f8-aa61-91515b181e27 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.377411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 205/421] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:18.380347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.380645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.380922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.410018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:18.411235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:18.422219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:18.422876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.423151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.423419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.425562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:18.427026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:18.429202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 217/422] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:18 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:18.435503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:18.573727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 206/423] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:18.620281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ebd83a-d189-4836-89f3-fad30f2aedf9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:18.626285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:18.681963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87599d-c03c-4c5b-b471-d3338c3c24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:18.682640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87599d-c03c-4c5b-b471-d3338c3c24c2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.682913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87599d-c03c-4c5b-b471-d3338c3c24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.683182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87599d-c03c-4c5b-b471-d3338c3c24c2 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.684572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87599d-c03c-4c5b-b471-d3338c3c24c2 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:18.686065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87599d-c03c-4c5b-b471-d3338c3c24c2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:18.688190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 218/424] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:18 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:18.695590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:18.697658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e18cd-4693-4887-96aa-f16855169d82 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.697942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e18cd-4693-4887-96aa-f16855169d82 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.698217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e18cd-4693-4887-96aa-f16855169d82 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.699846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 219/425] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:18.706895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.707166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.711983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.745853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:18.747976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:18.865387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 220/426] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:18.897751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:18.905274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66a674-ecb6-42bc-9853-6e0b4836b45d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:18.947420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:18.983637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:18.984263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:18.984534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:18.984799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:18.985565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=86c6007706a043bfa43e23a20cdbe653) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:18.985866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:18.991056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:19.013252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 207/427] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:18 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 611 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:19.109693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670772-3dd5-42a8-9c70-a9cccb70cca4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:19.112629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:19.151951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:19.152511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:19.152785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:19.153080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:19.164092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:19.166005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:19.171543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:19.172721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:19.173581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 221/428] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:19.182491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:19.402579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c54e6-7746-4656-ac43-f8458c185a82 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:19.481143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a46b5-0e49-4d74-8cfb-a73794a47820 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:19.487753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:19.488599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:19.489085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:19.489565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:19.490869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:19.491374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:19.494166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:19.515437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 222/429] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:15:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:19.521743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:19.542078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:19.542931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:19.543282 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:19.543784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:19.570431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:19.571384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:19.587985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:19.588962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:19.589804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:19.590410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 208/430] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:19.598096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:19.641634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:19.703975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a2f7d-fd0a-471a-b7de-5f03661995de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:19.766017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:19.766645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:19.766974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:19.767253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:19.786428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f48c5-1ffe-461b-a69f-4fec40bda3ca None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:19.847613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:19.850065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:19.860672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:19.861776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:19.862418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:19.862903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:19.864274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:19.864823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:19.867795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:19.886137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:19.895226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 209/431] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:19.902544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:20.009066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 223/432] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:15:20.014609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:20.053610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:20.113773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a3029-0de4-407e-8802-11d89e0503e8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:20.180779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6589f3-d79a-4b7b-a3ed-886f658c52de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:20.196643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:20.197485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.197632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.197965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.223464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:20.252776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:20.254163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.254818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.255312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.257712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:20.262201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:20.266013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 224/433] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:20 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:20.271683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:20.314189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:20.316239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:20.434026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:20.469304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 210/434] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:15:20.471805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:20.488787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189695b-55a3-4b0d-bffe-e365736f07ab None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:20.555153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af736742-bb8e-442c-a030-83c4929153a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:20.556158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af736742-bb8e-442c-a030-83c4929153a7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.556650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af736742-bb8e-442c-a030-83c4929153a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.557161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af736742-bb8e-442c-a030-83c4929153a7 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.559251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af736742-bb8e-442c-a030-83c4929153a7 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:20.562045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af736742-bb8e-442c-a030-83c4929153a7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:20.565627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 225/435] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:20 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:20.567983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0cd1-bb30-44eb-912d-085dc50bbbdc None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.568461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0cd1-bb30-44eb-912d-085dc50bbbdc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.568910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0cd1-bb30-44eb-912d-085dc50bbbdc None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.573058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 226/436] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:20.573622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0cd1-bb30-44eb-912d-085dc50bbbdc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:20.607436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387ff92-02eb-441c-a39b-903b5fbc0baf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:20.653270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b40a-88a3-404b-8aef-592c85c3a3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:20.654670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b40a-88a3-404b-8aef-592c85c3a3cb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.655146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b40a-88a3-404b-8aef-592c85c3a3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.655637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b40a-88a3-404b-8aef-592c85c3a3cb None admin] PATH_INFO: `/v3/projects/86c6007706a043bfa43e23a20cdbe653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.657820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6b40a-88a3-404b-8aef-592c85c3a3cb None admin] RBAC: Authorizing `identity:get_project(project_id=86c6007706a043bfa43e23a20cdbe653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:20.660513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6b40a-88a3-404b-8aef-592c85c3a3cb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:20.664062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 211/437] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:20 2022] GET /identity/v3/projects/86c6007706a043bfa43e23a20cdbe653 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:20.666273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.666759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.667228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.669909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:20.698351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:20.700417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:20.744315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0cd1-bb30-44eb-912d-085dc50bbbdc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:20.770312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:20.807586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 212/438] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:20.808618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:20.820409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:20.821703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:20.822374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:20.822995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:20.824482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=5ccb31a782a44a4aad62b2f6bba3d582, scope.project.id=1950f1d110d94df48b26b19d33e527ed) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:20.825382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:20.828336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759639-145a-4fe2-bdb9-b7bda826949b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:20.840573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 227/439] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:20 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=5ccb31a782a44a4aad62b2f6bba3d582 => generated 611 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:21.021895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc3686-2d83-4585-9f9b-b8297b6bf400 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.074673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afba6e-9116-4eb2-b06f-4381cfef3391 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.074902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afba6e-9116-4eb2-b06f-4381cfef3391 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.075135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afba6e-9116-4eb2-b06f-4381cfef3391 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.076809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 228/440] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:21.082858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.083158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.083460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.091850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:21.092559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.092837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.093197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.094486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:21.097046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:21.098648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 213/441] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:20 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:21.109204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:21.117589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:21.120356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:21.201558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:21.233099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 229/442] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:21.288926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:21.298403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3574b3-bebe-40d3-b9db-dc34b3311dc7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.358178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:21.359264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.359828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.360327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.361778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=86c6007706a043bfa43e23a20cdbe653) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:21.362836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:21.365636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:21.368448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:21.378271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 214/443] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:21 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=86c6007706a043bfa43e23a20cdbe653&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:21.433631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc02c-28b9-4a67-acbb-b7bb94d74e1a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.482171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:21.482991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.483445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.483846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.498129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:21.504110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:21.506591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:21.508158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:21.513568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:21.513658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 230/444] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:21.516929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:21.653787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad480-dea0-47e2-b3fb-45097c5196ee None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.659909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7c4f-e684-4bb6-9802-dd474ab13735 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.670963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:21.683966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:21.708363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:21.709039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.709338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.709614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.716416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:21.717550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.718067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.718570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.719908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:21.720305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:21.724215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:21.724416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:21.729096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:21.731695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:21.734335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 215/445] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:21.739842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:21.744150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 231/446] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:21.753202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:21.910576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cd3d6-faa1-4ea2-b3fd-5078b790cd33 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.938288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd3778-fb64-4576-9c42-cf4d28910f2c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:21.986211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:21.987115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:21.987551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:21.988454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:21.989033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:21.991162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:21.993226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.003227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.004199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.004647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.005093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.006718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 216/447] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:22.014500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.074965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.086052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.128585 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.131061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.187300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e270905-7164-4937-91db-5d3246a77dc5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:22.229996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.230739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.231050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.231486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.288204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 232/448] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:15:22.294764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.327505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.330023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.379028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.406318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.437256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92ca41-c730-4a4f-b0bc-30266ea2d6df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:22.465838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 217/449] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:22.472341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.485894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.486885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.487999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.488447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] PATH_INFO: `/v3/projects/1950f1d110d94df48b26b19d33e527ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.490391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] RBAC: Authorizing `identity:get_project(project_id=1950f1d110d94df48b26b19d33e527ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.492558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.494793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 233/450] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/projects/1950f1d110d94df48b26b19d33e527ed => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:22.501071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.604760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b088238-9737-4e46-904d-bc303f6943de None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:22.637452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591c509-bcf8-43bf-b117-6e6b7c6fe1fe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:22.650268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.651151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.651506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.651843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] PATH_INFO: `/v3/projects/bbb8bdae345348eca7be88e01d7954f8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.653795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] RBAC: Authorizing `identity:get_project(project_id=bbb8bdae345348eca7be88e01d7954f8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.655567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.657749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 218/451] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/projects/bbb8bdae345348eca7be88e01d7954f8 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:22.666955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.670325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.684980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.685654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.685928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.686199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] PATH_INFO: `/v3/groups/29c2dc8308944058901de5ade635d264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.687461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] RBAC: Authorizing `identity:get_group(group_id=29c2dc8308944058901de5ade635d264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.688952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.690705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 234/452] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/groups/29c2dc8308944058901de5ade635d264 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:22.697140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.713506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.808609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f5377-b805-4a41-a369-0fb302c1c6cb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:22.830217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15497a8b-63e8-443b-b75e-018230766795 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:22.854095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.854897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.855337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.855722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] PATH_INFO: `/v3/groups/a8c31117c90a4a239ea8ae11f30e8fe2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.857154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] RBAC: Authorizing `identity:get_group(group_id=a8c31117c90a4a239ea8ae11f30e8fe2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.858803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.860800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 219/453] 10.222.0.85 () {56 vars in 1180 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/groups/a8c31117c90a4a239ea8ae11f30e8fe2 => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:22.867509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:22.874091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:22.874710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:22.874991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:22.875288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:22.876025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] RBAC: Adding query filter params (group.id=29c2dc8308944058901de5ade635d264, role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=1950f1d110d94df48b26b19d33e527ed) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:22.876331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:22.877813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26197603-1b8b-4753-ab3d-538024702a30 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:22.905031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 235/454] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/role_assignments?group.id=29c2dc8308944058901de5ade635d264&scope.project.id=1950f1d110d94df48b26b19d33e527ed&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:22.955668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:22.995374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a5fd-f551-41dc-a054-90038d1a9d61 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:23.038199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:23.038820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:23.039096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:23.039366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:23.040117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] RBAC: Adding query filter params (group.id=a8c31117c90a4a239ea8ae11f30e8fe2, role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=bbb8bdae345348eca7be88e01d7954f8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:23.040421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:23.041974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6c023-7947-4a68-a83b-fda938df794c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:23.049278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 220/455] 10.222.0.85 () {56 vars in 1376 bytes} [Wed Nov 2 12:15:22 2022] GET /identity/v3/role_assignments?group.id=a8c31117c90a4a239ea8ae11f30e8fe2&scope.project.id=bbb8bdae345348eca7be88e01d7954f8&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:26.324005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40464b-8313-4393-9f39-3692195ebb66 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:26.324787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40464b-8313-4393-9f39-3692195ebb66 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:26.324787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40464b-8313-4393-9f39-3692195ebb66 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:26.326096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 236/456] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:26.333595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f491-a057-45c3-ba99-137e9e5de5bc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:26.334117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f491-a057-45c3-ba99-137e9e5de5bc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:26.334516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f491-a057-45c3-ba99-137e9e5de5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:26.370382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3135f491-a057-45c3-ba99-137e9e5de5bc None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:26.372493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3135f491-a057-45c3-ba99-137e9e5de5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:26.494796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 221/457] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:26.859165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40464b-8313-4393-9f39-3692195ebb66 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:26.882511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713331dd-f549-44ce-aa2f-7f9ea3b70183 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:26.883045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713331dd-f549-44ce-aa2f-7f9ea3b70183 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:26.883470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713331dd-f549-44ce-aa2f-7f9ea3b70183 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:26.885284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 222/458] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:26.896724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ba89-603f-412f-9a46-28cacf8a07d3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:26.897389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ba89-603f-412f-9a46-28cacf8a07d3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:26.897861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3ba89-603f-412f-9a46-28cacf8a07d3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:26.899759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 223/459] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:15:26.902343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc832d5-5480-4c17-9fd8-4d68b5ae3724 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:26.902909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc832d5-5480-4c17-9fd8-4d68b5ae3724 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:26.903379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc832d5-5480-4c17-9fd8-4d68b5ae3724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:26.923987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ccc832d5-5480-4c17-9fd8-4d68b5ae3724 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:26.925614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ccc832d5-5480-4c17-9fd8-4d68b5ae3724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:26.980193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:27.021771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:27.025306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40464b-8313-4393-9f39-3692195ebb66 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:27.038784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 224/460] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:27.041515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205179be-8819-4193-9253-3ba00a22889f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.042340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205179be-8819-4193-9253-3ba00a22889f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.043109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205179be-8819-4193-9253-3ba00a22889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.080792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-205179be-8819-4193-9253-3ba00a22889f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:27.084177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-205179be-8819-4193-9253-3ba00a22889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:27.106234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:27.107381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.107670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.107934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.133243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:27.135018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:27.151949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:27.152939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:27.153818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:27.153818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:27.153818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:27.156027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 237/461] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:27.163911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:27.202401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 225/462] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:27.348208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70f1ee-0de6-457b-b1aa-c49125b38174 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:27.409250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:27.430012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:27.431046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.431490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.431972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.433070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:27.433542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:27.435807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:27.463216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205179be-8819-4193-9253-3ba00a22889f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:27.472258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 238/463] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:27.479171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:27.609195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2366b0c-3d6d-431e-aabe-b53faa5da121 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:27.654649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:27.655439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.655886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.656334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.657252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:27.659432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:27.697148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 239/464] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:15:27 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:15:27.698457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:27.700839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205179be-8819-4193-9253-3ba00a22889f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:27.755668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:27.771705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:27.782082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:27.783330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.783840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.784481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.806791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:27.810321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:27.839890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aff5c-e15c-4baf-8d55-1895c7d3b14a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c49b0ad3-7124-4ed8-9e9a-0277711c48a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:27.841720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:27.842616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:27.843481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:27.843481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:27.843481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:27.844291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 226/465] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:27.846614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044055-a27b-4e65-abb3-7662c83ecd4f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.847124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044055-a27b-4e65-abb3-7662c83ecd4f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.847617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044055-a27b-4e65-abb3-7662c83ecd4f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.850891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 227/466] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:27.852398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13044055-a27b-4e65-abb3-7662c83ecd4f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:27.891858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:27.892508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.892783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.893081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.909531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:27.912485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9cc066f2-35a2-4893-a18e-8c465c7baea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:27.930599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:27.932363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 240/467] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f513a4-a9af-4505-b05f-c0d42fd0700c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f513a4-a9af-4505-b05f-c0d42fd0700c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:27.933951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f513a4-a9af-4505-b05f-c0d42fd0700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:27.954579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-24f513a4-a9af-4505-b05f-c0d42fd0700c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:27.955792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-24f513a4-a9af-4505-b05f-c0d42fd0700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:28.012135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13044055-a27b-4e65-abb3-7662c83ecd4f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:28.054941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:28.055854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.056311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.056777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.057667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:28.057950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 241/468] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:28.058339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:28.059896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66d5c1-c627-4843-8d65-880650dc725b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:28.060093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca84d1-9ec7-4ebf-99a5-e8d56f422bbb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.060587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca84d1-9ec7-4ebf-99a5-e8d56f422bbb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.061096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca84d1-9ec7-4ebf-99a5-e8d56f422bbb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.063615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 242/469] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:28.064760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca84d1-9ec7-4ebf-99a5-e8d56f422bbb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.087556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 228/470] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:28.090072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.090561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.091035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.102433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:28.115402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:28.116617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:28.154021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:28.203394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 229/471] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:28.204554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.250186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca84d1-9ec7-4ebf-99a5-e8d56f422bbb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:28.340401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:28.341441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.341910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.342372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.350560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:28.351121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:28.352309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:28.371205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 243/472] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:28.378295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.440316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bee530-8863-4a9b-9e62-47586d41d537 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:28.508596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:28.511174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:28.512343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.513480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.514005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.515297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:28.519352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:28.527549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95639eca-16c8-4215-b5db-db0466019247 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:28.566287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 230/473] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:15:28 2022] POST /identity/v3/users => generated 248 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:15:28.566287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.592690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:28.593334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.593664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.593937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.594644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:28.596178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:28.610657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:28.630795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 244/474] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:15:28 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:15:28.632222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.781757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89173-7f20-4517-9888-9dfb391e2115 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:28.832677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467a7bb-215d-4843-9787-d2050ec6aba5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:28.882554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:28.883776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.883968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:28.884272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.884729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.885255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:28.885784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:28.886270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:28.887231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:28.888932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:28.900898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 12:15:28.900898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 12:15:28.900898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [parameters: {'id': 'd36de5cfcb8942d797c73ef613bb1bde', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 12:15:28.900898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 12:15:28.908684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:28.911989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Nov 02 12:15:28.928007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 12:15:28.928967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 12:15:28.929927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 12:15:28.930810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 12:15:28.931714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 12:15:28.932544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'id': 'd36de5cfcb8942d797c73ef613bb1bde', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:28.933358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 12:15:28.934163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:28.938246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 245/475] 10.222.0.85 () {60 vars in 1131 bytes} [Wed Nov 2 12:15:28 2022] POST /identity/v3/roles => generated 144 bytes in 307 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:28.941897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:28.944337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:28.945500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:28.945500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:28.945500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:28.945500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 231/476] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:28.946758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.950555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:28.961866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:29.049416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:29.077383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51234aeb-b327-41be-a44b-3fca70befaf4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:29.082690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed2b1-74d8-4379-b679-9db2c909d27f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:29.122412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:29.123408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:29.123902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:29.124023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:29.124486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:29.124592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:29.124795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:29.125083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:29.125860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:29.126387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:29.132786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:29.141051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:29.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:29.143881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:29.146068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 246/477] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:15:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:29.150202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:29.178592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 232/478] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:29.189363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:29.307890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:29.336676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc195-d4a7-4578-b9ea-52ecb76e0e99 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:29.373602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c73a9-5978-4789-bb4a-99a682db81e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:29.374214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c73a9-5978-4789-bb4a-99a682db81e5 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:29.374489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c73a9-5978-4789-bb4a-99a682db81e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:29.374826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c73a9-5978-4789-bb4a-99a682db81e5 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:29.375500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c73a9-5978-4789-bb4a-99a682db81e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:29.377053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c73a9-5978-4789-bb4a-99a682db81e5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:29.390826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:29.405189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 233/479] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:15:29 2022] POST /identity/v3/users => generated 249 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:15:29.420072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0398f-e8da-42b8-a631-d3c8ec264f2b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:29.459945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:29.460567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:29.460846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:29.461140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:29.461811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:29.462109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:29.463489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ebe1e-eaeb-4d4a-b550-5dbeca0047c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:29.473096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 247/480] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:31.134128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999cd6-e50f-43e8-bc58-031490d1add0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:31.135014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999cd6-e50f-43e8-bc58-031490d1add0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:31.135014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999cd6-e50f-43e8-bc58-031490d1add0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:31.136707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 234/481] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:31.143430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:31.144195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:31.144195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:31.170837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:31.172177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:31.260801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 248/482] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:31.838364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999cd6-e50f-43e8-bc58-031490d1add0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:31.989923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999cd6-e50f-43e8-bc58-031490d1add0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:32.044623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c139214-734c-470e-b101-6e50ce49b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:32.045287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c139214-734c-470e-b101-6e50ce49b90d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.045560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c139214-734c-470e-b101-6e50ce49b90d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.045841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c139214-734c-470e-b101-6e50ce49b90d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.061665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c139214-734c-470e-b101-6e50ce49b90d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4c139214-734c-470e-b101-6e50ce49b90d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:32.066952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:32.069198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:32.070803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 235/483] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:32.081444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:32.177401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:32.219519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef8e8d-404b-43e7-8124-685cc048fd0e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.220033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef8e8d-404b-43e7-8124-685cc048fd0e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.220460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef8e8d-404b-43e7-8124-685cc048fd0e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.223374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 236/484] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:32.232070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c939350-6b49-42b2-b120-5a76f723b53c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.232558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c939350-6b49-42b2-b120-5a76f723b53c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.233062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c939350-6b49-42b2-b120-5a76f723b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.239459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41a8d7-f33d-4a9a-a0b5-35b6c72b48c1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:32.265322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8c939350-6b49-42b2-b120-5a76f723b53c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:32.267404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8c939350-6b49-42b2-b120-5a76f723b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:32.279975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:32.280818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.281292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.281702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.282508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:32.282998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:32.284573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:32.294068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 249/485] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:32.302032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:32.367864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:32.409590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 237/486] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:32.561775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5dcc1-de6b-4a27-b4e9-641e2082fc57 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:32.597505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9884c80-e1c8-4a6d-8d12-13bf84229b6c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.598085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9884c80-e1c8-4a6d-8d12-13bf84229b6c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.598355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9884c80-e1c8-4a6d-8d12-13bf84229b6c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.600037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 238/487] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:32.610486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:32.615415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62502-77ea-4ae9-92f6-21785c2a6ff7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.615710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62502-77ea-4ae9-92f6-21785c2a6ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.615953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62502-77ea-4ae9-92f6-21785c2a6ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.651244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:32.652777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.653631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.654437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.668444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7fc62502-77ea-4ae9-92f6-21785c2a6ff7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:32.674701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7fc62502-77ea-4ae9-92f6-21785c2a6ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:32.687835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:32.690851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:32.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:32.719162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:32.720774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:32.720774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:32.720774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:32.724223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 250/488] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:32.735396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:32.834326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 239/489] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:32.836567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc763de-2a84-4096-9d8c-49844898eab1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.837094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc763de-2a84-4096-9d8c-49844898eab1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.837576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc763de-2a84-4096-9d8c-49844898eab1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.839934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 240/490] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:32.848709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401efced-c685-4ebb-9cc7-d7598c65acaf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:32.849198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401efced-c685-4ebb-9cc7-d7598c65acaf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:32.850225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401efced-c685-4ebb-9cc7-d7598c65acaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:32.881963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-401efced-c685-4ebb-9cc7-d7598c65acaf None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:32.883968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-401efced-c685-4ebb-9cc7-d7598c65acaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:32.938523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0940c84-7835-4227-85d2-e663c6274a58 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.003368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 241/491] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:33.004723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034298-a0a5-4a1d-a644-738526b7b410 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.005171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034298-a0a5-4a1d-a644-738526b7b410 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.005576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034298-a0a5-4a1d-a644-738526b7b410 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.007143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 242/492] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:33.007704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3034298-a0a5-4a1d-a644-738526b7b410 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.015094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.016090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.016753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.017105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.017837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:33.018213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:33.019827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8aa13-9788-4f5b-9309-3ebc75c8ee70 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:33.037447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 251/493] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:33.037447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.037447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.037447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.056652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:33.058034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:33.063554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.089883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.149219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 252/494] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:33.150959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.154242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3034298-a0a5-4a1d-a644-738526b7b410 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.223708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.224893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.228337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.228680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.243110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:33.248253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:33.250178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 243/495] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:33.251751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.346479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d6c50-f4fb-4ede-83b6-09f8641123d1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.370544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.374482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a98c-59cc-4c94-92db-b19f11cecdd9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.386343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.387067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.387253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.387533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.398258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:33.400169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:33.414596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c278b0-c969-4f40-9901-19641712675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:33.415005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:33.416602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:33.418300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:33.419219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c278b0-c969-4f40-9901-19641712675f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.419219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c278b0-c969-4f40-9901-19641712675f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.419219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c278b0-c969-4f40-9901-19641712675f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.419595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 253/496] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:33.419595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.435284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c278b0-c969-4f40-9901-19641712675f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-80c278b0-c969-4f40-9901-19641712675f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:33.440187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:33.442243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 244/497] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:33.443377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c278b0-c969-4f40-9901-19641712675f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.464425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.588190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d4a42-e63f-48c0-bd9f-1916b4a3b228 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.595176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c278b0-c969-4f40-9901-19641712675f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.630062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.630606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.631494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.631772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.632459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:33.632764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:33.634230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:33.635135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.635776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.636051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.636312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.644569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 254/498] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:33.645642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.648525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:33.653700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:33.655478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 245/499] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:33.657231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.685293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.757121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.774876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f00daf-a7cf-4a8a-8bda-07bf31642b3a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.777576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebaccf2-ed76-4071-9412-21b6baae0c10 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.815217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.815978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.816339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.816710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.817619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:33.818008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:33.818356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:33.819464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:33.819735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:33.819996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:33.820546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:33.820705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:33.820942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:33.822376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:33.831504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 246/500] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:33.832379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.861454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 255/501] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:33.862631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:33.930436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:33.954921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d7cf0-c8cd-425e-b3a7-c045695089de None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:33.979157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dec52-2696-4748-be2a-9f84e5ca0006 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.007094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.007521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.007796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.008061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.023713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.024350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.024627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.024898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.025335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.027904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.036358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.038384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.040383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:34.045373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:34.047270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:34.048913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:34.048913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.048913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:34.048913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 247/502] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:34.048913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:34.062994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:34.064830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:34.066546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:34.066546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.066546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:34.066546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 256/503] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:34.066972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.170922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6746f-4d93-4250-a985-005a6e0a2cd9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.193576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414b15a-0048-4880-8ac8-8a0cd4ab3577 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.225326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.226060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.226363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.226640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.227390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:34.227689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.228633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.229288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.229380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.229566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.229910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.230948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.241356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 248/504] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:34.241607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.247574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.249443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:34.265328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:34.267315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:34.269278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:34.269278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.269278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:34.269278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 257/505] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:34.269278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.328352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.374607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7620203-6461-42e2-abb8-785021f7bbf7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.394576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0c4ff-9bff-4af3-90d4-07d38b4b9a2e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.420185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.421246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.421747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.422230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.434396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.435444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.435953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.436447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.437675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:34.438183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.440750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.467277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 258/506] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:34.467277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.512805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.520297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.522348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.613597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf0879-3b87-42d7-bfa3-4fb260bbec51 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.622093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.648671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 249/507] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:34.648671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.653825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.654899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.655758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.656511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.658033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:34.658496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.660179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.677280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 259/508] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:34.678351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.751334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.758905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eab528-b8ce-4c72-a3fc-04b9ca017b48 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.796801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bd538-b0a0-4274-a595-a9169854ff00 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:34.798627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.799270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.799540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.799793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.811270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.812727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:34.828685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:34.829832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:34.830948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:34.830948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:34.830948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:34.830948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 250/509] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:34.830948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.844856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:34.845584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:34.845842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:34.846105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:34.846760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:34.847060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:34.848433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:34.861935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 260/510] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:34.863387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:34.905440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:34.956015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d7d9-2daa-4303-ac15-5e4010efa763 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.000881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce7608-b8c0-42dc-bbd7-2875ae244587 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.006835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.007802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.008170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.008575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.009785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.012555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.040014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.040695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.040979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.041277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.043103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 251/511] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:15:34 2022] POST /identity/v3/users => generated 247 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:15:35.044708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.056505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.059099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.061053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:35.088753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:35.089938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:35.090818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:35.090818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:35.090818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:35.090818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 261/512] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:35.092135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.190782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522e864-b327-492e-a0a8-d689faa81201 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.228633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:35.241249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.241999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.242312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.242552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.255438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.257639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.266517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db355f0-be9d-4772-945d-8054a1993aca None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:35.276144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:35.277172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 12:15:35.278156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:35.278666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 252/513] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:35.280000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.326760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.327879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.328342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.328903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.330211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:35.330831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.333594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.354478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 262/514] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:35.355458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.364201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:35.408533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae570427-7521-4a62-9d31-26f7cd8fe207 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.452524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.453186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.453470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.453740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.462985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.464481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:35.483638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:35.484814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:15:35.485938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:35.493251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 253/515] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:35.493251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.534194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae13823-e9a1-4060-ab77-fd6407e446d1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.602704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.603948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.604392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.604840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.605792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:35.606294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.608643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.622283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:35.640981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 263/516] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:35.642569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.664862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61add67a-57f5-4a1f-9f50-6490f478c79d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.712647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:35.734252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.735297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.735816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.736990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.738628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:35.739263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.742687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.792517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 254/517] 10.222.0.85 () {56 vars in 1137 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:35.793447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.806887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2eb2b-3f90-4d11-87cc-6ba18e462b81 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.855669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.856343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.856628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.857104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:35.867576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:35.869038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:35.891478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:35.898541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 264/518] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:35.900241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:35.954913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f08ad-0abb-4c18-a246-c2e1fcd73e65 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:35.972505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:35.998385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:35.999102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:35.999377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:35.999641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.000404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:36.000702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:36.002241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:36.033369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 255/519] 10.222.0.85 () {56 vars in 1137 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:36.033369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:36.051315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9946d8-5694-4827-a959-19cc45e9ce72 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:36.084665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:36.095012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:36.095761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:36.096040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:36.096298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.103758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:36.105347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:36.119581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:36.121436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 265/520] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:36.123395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:36.166333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a7f86-51df-481b-90c1-69a7e20cfa30 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:36.207711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:36.208910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:36.209213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:36.209627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.281893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883631-6a5b-4d2a-8877-c18e0bf8bbb3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:36.301129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:36.302796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:36.318508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:36.351740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:36.352482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:36.353351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:36.354262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:36.354773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.356281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:36.357011 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:36.360981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:36.388013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 266/521] 10.222.0.85 () {56 vars in 1139 bytes} [Wed Nov 2 12:15:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:36.389564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:36.443569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 256/522] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:36.443569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:36.543054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3818adc-71e7-47e8-b97d-4a01aa149cdb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:36.600080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:36.601225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:36.601501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:36.601769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.609904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c784ff-d358-4029-b497-03c1f9ed72dd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:36.653115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:36.653731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:36.654004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:36.654267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.654964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:36.655266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:36.656275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:36.656799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:36.673179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:36.685271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 257/523] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:36.685271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:36.703486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:36.705860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:36.832297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2a813-6065-4c0f-b96b-f5bfbfdb392c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:36.877462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 267/524] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:36.877462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:36.887728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:36.888354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:36.888638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:36.888897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:36.990465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:36.997279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:37.003187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.005191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.026160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583734e1-8f10-4997-80c1-b9ca90a8002f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.079068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.079753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.080227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.080841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.099828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.103439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:37.131430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:37.132970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:37.133878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:37.133878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:37.133878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:37.134055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 268/525] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:37.135597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.172698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 258/526] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:37.172698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.285745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09111f9c-a0ec-491a-aed6-0f4f015d8dd6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.317452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:37.321066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:37.326245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bbf02-2e02-4e24-955d-e676ea564b7f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.334353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.335598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.335933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.336206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.337420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=c173404c311b43e29193004dc41480b8) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:37.337903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.339935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.347714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 269/527] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/role_assignments?user.id=c173404c311b43e29193004dc41480b8&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:37.348602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.377204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.377872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.378266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.378541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.391575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.393079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:37.418063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:37.419817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:37.422005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:37.422005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:37.422005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:37.422005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 259/528] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:37.422005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.528098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d643277-19c0-4034-a6fa-91d5afcdf1fb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.576565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.577597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.577597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.577856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.578401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:37.578701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.580150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420bdf8-ed4e-46b5-9c07-e8bdc2cbcf7a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.598769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d4ad-3e42-41c3-97bd-b406b0c00c0f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.606089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 270/529] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:37.606089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bddc51-c642-4ebd-b968-ce8572204b7e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.606089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bddc51-c642-4ebd-b968-ce8572204b7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.606089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bddc51-c642-4ebd-b968-ce8572204b7e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.607795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 271/530] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:37.612006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bddc51-c642-4ebd-b968-ce8572204b7e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.645426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.646129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.646483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.646822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.659100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.660674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:37.686516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:37.687696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:37.688695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:37.688695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:37.688695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:37.694601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 260/531] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:37.695618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.702219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:37.703336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:37.761994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bddc51-c642-4ebd-b968-ce8572204b7e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.810150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.811112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.811589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.811934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.812646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:37.813185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.814492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313a3ca-1d63-4950-8c32-7074d8dd7ffd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.833318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 272/532] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:37.837084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.837577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.838059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.847193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4133612-b26a-459f-b454-8caabb7c2ab3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:37.867592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:37.868813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:37.898851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:37.899959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:37.900664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:37.901404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:37.914720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:37.916701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:37.942908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:37.945497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:37.947403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:37.950168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 261/533] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:37.950915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:37.965638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 273/534] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:37.965638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.036029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:38.047783 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:38.096207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297214f4-83f2-4a5b-b699-1d431278520e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.124522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf9fc1-3b13-411f-9bb0-70f6e2457f20 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.137555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.138520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.139004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.139469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.140290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:38.140763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.142357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.158841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 262/535] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:38.160383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.183315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.184731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.185271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.185765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.200003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.203333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:38.227500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:38.228414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:38.229378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:38.231949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 274/536] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:38.233301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.314454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7263-4610-4f23-846f-925daa98e087 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.349198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:38.366015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.367158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.367431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.367697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.368397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:38.368726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.370990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.377196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff101ddf-f1a0-4e0a-82bb-a9e7dee9b868 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.379351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:38.393025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 263/537] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:38.394451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.435119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.435720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.435991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.436257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.443709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.445492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:38.458097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:38.464710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 275/538] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:38.466261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.533283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727f9c4-a314-4c49-9158-e058a3060263 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.574208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.575224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.575725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.576225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.577600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:38.578248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.581041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.605887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 264/539] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:38.614373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.615153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255114-6833-4202-aea1-e0c598719c1f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.657912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.659040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.659814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.660242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.676542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:38.686317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:38.687309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:38.688352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:38.689397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:38.696182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 276/540] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:38.696182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.722935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:38.796475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f58262-01b3-4a88-85de-b701540509cd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.813520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84aeb0-997c-45ab-ab3c-06378a189cef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:38.837826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.838630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.838896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.839163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.839911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=ce6a8adf15e3498d95442b3ad0ef3864) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:38.840258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.842003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.849393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 265/541] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/role_assignments?user.id=ce6a8adf15e3498d95442b3ad0ef3864&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:38.849550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:38.865467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:38.866734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:38.867389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:38.868010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:38.869044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:38.869801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:38.872042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:38.901943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 277/542] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:38.904308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:39.029351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:39.060211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08718e-e00d-4c3b-8bf3-df2bca3e60a2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:39.065233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884c6b1-8431-4067-a871-965b913435ee None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:39.084159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:39.113825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:39.114850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.115319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.115886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.116966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:39.117554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:39.119242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:39.119562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:39.119972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.120332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.120669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.121495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=b840491b8d05489c8475e1125dbc0b5b) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:39.122058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:39.123970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:39.131383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 278/543] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:39.132664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:39.141056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 266/544] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:38 2022] GET /identity/v3/role_assignments?user.id=b840491b8d05489c8475e1125dbc0b5b&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:39.141056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:39.309014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e26317-ce79-463f-b6fe-deff0d01a404 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:39.357250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:39.358774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.359247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.359746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.369846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4aed7-5765-448b-b255-21b566ad1d77 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:39.372173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:39.374051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:39.392691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:39.393712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:39.394554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:39.394554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:39.394554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:39.396743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 267/545] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:39.408438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:39.431303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:39.431934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.432217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.432482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.433268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=e102dacae430417aa150ddd53cf34f31) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:39.433573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:39.435052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:39.451770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 279/546] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:39 2022] GET /identity/v3/role_assignments?user.id=e102dacae430417aa150ddd53cf34f31&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:39.472134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:39.628614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215056ba-fe90-417d-93f2-b408d67d0f8c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:39.695503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:39.696115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.696390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.696664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.697552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:39.697860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:39.699538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0758c-9374-4023-825c-c6e83cf565fc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:39.723533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 268/547] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:39.729876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:39.749968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:39.750607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e8dad-4979-43f1-ac0a-edca1f223c09 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.751097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e8dad-4979-43f1-ac0a-edca1f223c09 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.752535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e8dad-4979-43f1-ac0a-edca1f223c09 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.756875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 269/548] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:39.767189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:39.768582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:39.769125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:39.805385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:39.808991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:39.957117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00673723-b93d-4ead-ab22-98481272bf92 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:39.986631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 270/549] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:40.045788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:40.046447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:40.046721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:40.046986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:40.055985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:40.057541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:40.073804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:40.074748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:15:40.076112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:40.082248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:40.082789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 280/550] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:40.105420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:40.271317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df3b34-748c-4971-a648-68801dc374bc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:40.334014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:40.334651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:40.334923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:40.335184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:40.335970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:40.336270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:40.337845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:40.363545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 271/551] 10.222.0.85 () {56 vars in 1135 bytes} [Wed Nov 2 12:15:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:40.369856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:40.445568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:40.487495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:40.555899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22304c1-0d14-41e6-b468-48a4d75b9b21 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:40.598201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:40.598827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:40.599188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:40.599362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:40.601802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:40.672605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826e82-5729-466b-8794-b2410ce28a4c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:40.736767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:40.738245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:40.738881 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:40.739152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:40.739415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:40.741823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:40.763974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:40.778426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:40.780230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:40.786440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 272/552] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:40.806158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:40.858479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:40.924982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 281/553] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:40.932838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:41.071182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d07f-1647-4441-a9bc-94d8b968f02b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:41.121198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968d486-9533-4d54-ae1e-7863ccc01843 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:41.158256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:41.175746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:41.177192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.177702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.177809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:41.178324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.178606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.178873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.179164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.180084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:41.180386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:41.181801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:41.194734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:41.197446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:41.201506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 273/554] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:41.211337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:41.225064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:41.226069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:41.227420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:41.227420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:41.227420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:41.229893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 282/555] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:41.235217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:41.289456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:41.366309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e578c7-cdfc-4d1e-93fd-74e13a0b878a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:41.407010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:41.408027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.408507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.408977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.409901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:41.410416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:41.412073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:41.426954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 283/556] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:41.434150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:41.488152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598ec47-b0fe-4b85-99d0-8acf40c54206 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:41.569056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:41.571179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.571716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.572254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.594281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:41.596265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:41.626891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:41.628842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:41.630098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:41.630098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:41.630098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:41.634028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 274/557] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:41.640489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:41.642573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:41.703618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:41.716938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e9a6e-3415-4497-8a41-d01ecd50e142 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:41.798204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:41.799279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.799780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.800130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.813105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:41.813974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09507de5-312d-4b24-abda-7f191bbe9853 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:41.815888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-da1db4ce-47f4-4b6c-a87f-9ecf51fd6f20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:41.837954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:41.838929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:41.839894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:41.847399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 284/558] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:41.849828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80ff5-cbc2-401e-b1f6-bf2a908e22ce None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.850343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80ff5-cbc2-401e-b1f6-bf2a908e22ce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.850830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80ff5-cbc2-401e-b1f6-bf2a908e22ce None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.853230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 285/559] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:41.854434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80ff5-cbc2-401e-b1f6-bf2a908e22ce None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:41.870473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:41.871132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.871602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.871953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.872627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:41.873046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:41.875197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e20ae1-58b0-4a99-9a8b-c3fa5ce559e8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:41.890451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 275/560] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:41.895541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39647-8114-4c67-8992-6d176b91614a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:41.895541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39647-8114-4c67-8992-6d176b91614a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:41.895541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39647-8114-4c67-8992-6d176b91614a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:41.916928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-06b39647-8114-4c67-8992-6d176b91614a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:41.918301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-06b39647-8114-4c67-8992-6d176b91614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:42.037204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 276/561] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:42.038532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39647-8114-4c67-8992-6d176b91614a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.042837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80ff5-cbc2-401e-b1f6-bf2a908e22ce None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:42.089119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:42.089742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.090065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.090339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.091341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:42.091648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:42.093374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:42.094259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d9e9c-b660-4fdf-8e51-1dd76e5a71b7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:42.112671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:42.119466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 286/562] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:42.121841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5ba54-e6eb-4a75-bd36-99c1bee67126 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.122458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5ba54-e6eb-4a75-bd36-99c1bee67126 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.123205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5ba54-e6eb-4a75-bd36-99c1bee67126 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.125431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 287/563] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:42.126939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5ba54-e6eb-4a75-bd36-99c1bee67126 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.179265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39647-8114-4c67-8992-6d176b91614a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:42.228478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:42.229140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.229963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.230231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.246267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:42.247733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3fba08bf-cb7c-4d93-9258-bbc7356f4a44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:42.260528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:42.261791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 277/564] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:42.263430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592752-f193-4687-a557-18873a264b21 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.264208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592752-f193-4687-a557-18873a264b21 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.264208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592752-f193-4687-a557-18873a264b21 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.269077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 278/565] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:42.269077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb4fc-835c-46b2-8c9e-789a6d4d255d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.269077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb4fc-835c-46b2-8c9e-789a6d4d255d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.269077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb4fc-835c-46b2-8c9e-789a6d4d255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.280711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5ba54-e6eb-4a75-bd36-99c1bee67126 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:42.288062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-352eb4fc-835c-46b2-8c9e-789a6d4d255d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:42.289577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-352eb4fc-835c-46b2-8c9e-789a6d4d255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:42.324414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:42.325446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.325725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.325992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.326715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=ef33e7dcb56a481aa051c09d45bfc83d) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:42.327019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:42.328429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:42.340903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 288/566] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/role_assignments?user.id=ef33e7dcb56a481aa051c09d45bfc83d&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:42.340903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.373861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 279/567] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:42.375521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.375799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.376070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.397986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:42.399677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:42.439423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:42.445841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:42.501651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 280/568] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:42.503252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.504500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8ded9-512b-4128-a199-222f828532cb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:42.553874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:42.554823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.554823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.554996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.555715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:42.556019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:42.557542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:42.593503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 289/569] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:42.602247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.699972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3506ace-9802-4c1a-858d-c4b0eb2dbb1c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:42.746758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:42.747420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.748280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.748555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.765612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:42.776229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:42.777305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:42.778164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:42.779386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 281/570] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:42.780557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.805549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb69d-85e8-4685-a252-45c9d14b5987 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:42.817537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:42.833258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:42.847950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:42.848597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:42.848869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:42.849165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:42.861782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:42.863252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:42.885695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:42.887494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:42.888604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:42.888604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:42.888604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:42.888604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 290/571] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:42.888604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:42.962681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409b13-3718-4ac6-b802-8036fdc39644 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.023835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.024863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.025174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.025682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.032118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6fc53-0c74-4072-bc43-d938912bb779 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.042521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:43.048764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:43.050138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 282/572] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:43.050996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.077589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.078209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.078487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.078759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.079431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:43.079734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.081143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.092251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 291/573] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:43.092485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.140156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.150439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.177662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0132-a23d-4c4d-b1d2-158d11f52c50 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.219182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.219778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.220038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.220303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.226728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5431f8-51a1-4ec4-868f-17e99ac86890 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.232388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:43.237235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:43.238181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:43.239025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 283/574] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:43.239711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.272857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.273635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.273910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.274795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.275478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:43.275780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.277183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.291465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 292/575] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:43.297876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.379248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69080b9-65a9-44fa-83c2-1e1d0deb684f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.417697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.418745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.419383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.419768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.419930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.420626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:43.420922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.422373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.425360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.427003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b258c-b152-4e39-8739-27a822cacf5d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.433359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 284/576] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:43.434384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.480881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.481564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.481872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.482105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.493334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.494779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:43.512195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:43.513244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:43.514806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:43.514806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:43.514806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:43.514806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 293/577] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:43.515078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.562014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882a3c0-e160-4a03-96f7-1ae9b992de8a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.599017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.599738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.600098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.600435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.601196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:43.601499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.603388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.612728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 285/578] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:43.613496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.633909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c6b91-5489-40f6-b2c4-e1af8cfbd532 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.672678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.673493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.673841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.674239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.679125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.683493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.684968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:43.699246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:43.700190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:43.701127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 294/579] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:43.702787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.717197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.731196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65737-f001-4ba7-803c-48e776d492c0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.769385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.770220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.770500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.770773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.782554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.784020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:43.800226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:43.801254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:43.802166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:43.802166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:43.802166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:43.802166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 286/580] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:43.802166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.823891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e128c-f5a0-436d-9ffa-6bfc5796d571 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.863006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.863617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.864158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.864434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.865677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:43.866119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.868218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.883205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 295/581] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:43.884158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:43.913495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60526d-e69f-4dac-8e2b-94a9f2ea0b16 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:43.927850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.952572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:43.953236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:43.953620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:43.953890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:43.958522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:43.967345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:43.968983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:43.985244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:43.986212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:43.987149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:43.987149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:43.987149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:43.987149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 287/582] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:43.987879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.003856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31424-3222-4394-b947-f950447d9c2f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.043697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.044304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.044664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.044872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.045599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:44.046116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.047295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.064141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 296/583] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:44.064141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.105270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35917c5c-6c1e-486d-8cb3-a47d7abbb771 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.143207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d85448-f020-4fff-969a-728557104279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.143831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d85448-f020-4fff-969a-728557104279 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.144107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d85448-f020-4fff-969a-728557104279 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.144385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d85448-f020-4fff-969a-728557104279 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.145069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d85448-f020-4fff-969a-728557104279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:44.145379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d85448-f020-4fff-969a-728557104279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.146812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d85448-f020-4fff-969a-728557104279 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.165157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 288/584] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:44.165157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d85448-f020-4fff-969a-728557104279 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.187778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3c53-01b7-4509-ab8d-ecc32d2e1136 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.191807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:44.220794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:44.230863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.231492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.231766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.232025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.240085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.241464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:44.253327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:44.254348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:44.255364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 297/585] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:44.260234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.277237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d85448-f020-4fff-969a-728557104279 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.313602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.314210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.314521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.314972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.315439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:44.315737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.317119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.334432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 289/586] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:44.334432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.371543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5009f-9ccd-4a0e-a1e3-990bde3157a5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.408074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.408736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.409038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.409287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/c173404c311b43e29193004dc41480b8/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.421648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:44.426578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=c173404c311b43e29193004dc41480b8, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.428038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.440317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9224342-0d43-44e0-b038-4206b50186b4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.442427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 298/587] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:15:44 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/c173404c311b43e29193004dc41480b8/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:44.454003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.489797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.491218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.492268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.493393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.507006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.509104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.513091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:44.533120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:44.534179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:15:44.535155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:44.538489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 290/588] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:44.540309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.613396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb5e45-6c09-4614-a4a2-4aee669bfa50 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.666334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.667378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.667877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.668361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.669499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:44.669843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.672607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.675698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be644e-56f9-426e-a5f1-02305572bfa8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.692430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 299/589] 10.222.0.85 () {56 vars in 1137 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:44.693132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.713961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.714598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.714877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.715153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.723329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.724784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8b878f8a-316d-491f-b1a4-a2621a59bf08 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:44.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:44.737772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:44.738774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 291/590] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:44.739439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee80087-1b69-471e-baf6-014f86c54b70 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.739439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee80087-1b69-471e-baf6-014f86c54b70 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.739621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee80087-1b69-471e-baf6-014f86c54b70 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.745769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 292/591] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:44.745769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee80087-1b69-471e-baf6-014f86c54b70 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.749623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:44.803375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:44.811039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c344c-5b58-4d9c-b382-62df68838d03 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.846315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.846973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.847187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.847452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.848185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:44.848487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.850287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.864274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee80087-1b69-471e-baf6-014f86c54b70 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:44.878035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 300/592] 10.222.0.85 () {56 vars in 1137 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:44.878035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:44.913876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:44.915255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.915536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.915809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.928079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:44.929718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-2f9267b0-76d1-4444-863f-e708ca356c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:44.948927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:44.950003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:44.950999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:44.950999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:44.950999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:44.950999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 293/593] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:44.951851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:44.951851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:44.952017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:44.971996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:44.974228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:45.008679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2c2ba-c896-4a6d-98e5-256f68c916d9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.044692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.049619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.050267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.050589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.051274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.052341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:45.052632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.054081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.065074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 294/594] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:45.065074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.080355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 301/595] 10.222.0.85 () {56 vars in 1139 bytes} [Wed Nov 2 12:15:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:45.081636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.119374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.219618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c2b7-b94f-4df7-b40d-26de241972fc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.242835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7608e6a-a2f5-419e-9469-65accda638fb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.278386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.279064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.279408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.279751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.286931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.287539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.287924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.288257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.288907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:45.289239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.291270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.292410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.293894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.304565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 302/596] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:45.305494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:45.311165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:45.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:45.313652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:45.313652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:45.313652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:45.316917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 295/597] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:45.317846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.359685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.416190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.432338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc3249-8f27-4257-be92-224c2355f6d7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.449055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639ab2a-1737-44e2-97fb-97fb817caa62 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.471689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.472803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.473116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.473394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.481450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.482983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.488046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.488718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.489012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.489294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:45.494804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:45.495701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:45.496651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:45.501034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 296/598] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:45.501695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.501885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.503131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:45.520824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:45.521968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:45.528843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:45.528843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:45.528843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:45.528843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 303/599] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:45.528843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.634314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.639861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b2e5-da8e-4508-a112-5869e67ed5a9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.655634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a425-87e7-4631-bae5-0fdf46eb53aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.682571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.696515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.697637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.697918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.698185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.705172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.706345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.706956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.707551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.708812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:45.709513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.709956 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:45.712587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:45.715896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:45.716868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:45.717744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 304/600] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:45.718428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.746852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 297/601] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:45.746852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.831678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049cafc-f5fd-4e6d-81cc-57d839f6450f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.885476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.885476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.885476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.885476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.886986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:45.886986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.887941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.901472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f812-ad2a-4442-95c8-43e17e14fca2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:45.902402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 305/602] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:45.908072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:45.954047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:45.954976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:45.955433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:45.955923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:45.956675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:45.957116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:45.959333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:45.966895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:45.986839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 298/603] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:45.986839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.012239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:46.050534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca87196-2030-464d-9551-16ff4ee2cc76 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.096653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.097351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.097708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.098077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.098841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:46.099154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.100514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.104505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d945a9c-1da6-4d07-b839-479a0d36050c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.115745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 306/604] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:46.115745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.151724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.152472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.152751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.153194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.162300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.163904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:46.182102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:46.184068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:46.186121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:46.187197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 299/605] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:46.192284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.261242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50524d40-f09f-4828-8fa3-8a48a56e1af2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.269169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:46.299945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:46.302339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.303214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.303493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.303769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/ce6a8adf15e3498d95442b3ad0ef3864/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.321594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=ce6a8adf15e3498d95442b3ad0ef3864, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.323046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.338168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bebc12-242e-4e4f-9a4e-9d4bbdf987bb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.341872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 307/606] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:15:46 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/ce6a8adf15e3498d95442b3ad0ef3864/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:46.343481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.419213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.420227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.420513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.420869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.433704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.435684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:46.467736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:46.468679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:46.469608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 300/607] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:46.474064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.507257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54130aab-6147-4dc4-9ce7-4f9b0224cf56 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.557590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.558365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.558548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.558775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.571373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.572530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:46.591388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:46.592784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:46.593833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:46.593833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:46.593833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:46.597129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 308/608] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:46.598634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.614948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:46.661514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2637de-305d-4071-8f0b-2467e12ff1c8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.722005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.723603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.724022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.724886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.726176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:46.726482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.728068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.736361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:46.748248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 301/609] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:46.748728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.755012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d06db0-337d-4a3e-9777-14b2457dc688 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.796083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.796698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.796971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.797280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.797975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:46.798284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.799701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52d70-b9f0-4f1f-a437-1b70d483134f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.818663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 309/610] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:46.818663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247502-6adf-4220-873b-425b23e6ec1a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.818663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247502-6adf-4220-873b-425b23e6ec1a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.818663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247502-6adf-4220-873b-425b23e6ec1a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.818663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 310/611] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:46.818663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247502-6adf-4220-873b-425b23e6ec1a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.889116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504e015-4f54-4e4a-93d3-ebc925484f89 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:46.938516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:46.940846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:46.941664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:46.942023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:46.942371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:46.943126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:46.943507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:46.945522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:46.965404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 302/612] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:46.965404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:46.987665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247502-6adf-4220-873b-425b23e6ec1a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:47.034273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e1f4b-1c43-434f-9823-8a9f4a387961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:47.035342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e1f4b-1c43-434f-9823-8a9f4a387961 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:47.035462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e1f4b-1c43-434f-9823-8a9f4a387961 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:47.035815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e1f4b-1c43-434f-9823-8a9f4a387961 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/b840491b8d05489c8475e1125dbc0b5b/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:47.042102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:47.055032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e1f4b-1c43-434f-9823-8a9f4a387961 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=b840491b8d05489c8475e1125dbc0b5b, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:47.057293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e1f4b-1c43-434f-9823-8a9f4a387961 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:47.081053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 311/613] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:15:46 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/b840491b8d05489c8475e1125dbc0b5b/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:47.081053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:47.081053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:47.081053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:47.113872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:47.115416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:47.132894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53e5d2-b63a-44d6-ac8b-890f94c498d5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:47.195718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:47.196727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:47.196926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:47.197296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/e102dacae430417aa150ddd53cf34f31/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:47.214564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=e102dacae430417aa150ddd53cf34f31, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:47.216185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:47.225701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 312/614] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:47.228123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:47.237786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 303/615] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:15:46 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/e102dacae430417aa150ddd53cf34f31/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:47.314598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:47.406370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf2636-ad12-4de5-9277-2a5d6b84b1d8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:47.445224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:47.446101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:47.446827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:47.447061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:47.457125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:47.458607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:47.484801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:47.485816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:47.486759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 313/616] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:47.494292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:47.639712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:47.674371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758fb64-29ef-49ac-a98d-04150ff0f450 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:47.719052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:47.792854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:47.794386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:47.794871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:47.795397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:47.796352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:47.796942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:47.798847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:47.832957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 304/617] 10.222.0.85 () {56 vars in 1135 bytes} [Wed Nov 2 12:15:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:47.842798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:47.857804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4f168-c209-449f-acd6-336ba6af4648 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:47.869525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:47.898201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:47.898841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:47.899128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:47.899416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:47.920014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:47.933473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:47.934595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:47.935892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 314/618] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:47.940922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.010737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576b7b4-e33f-472b-897b-be8ab2a1ad3e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.065999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201069-18f1-4043-bfa1-7e4ff8962970 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.068428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.069481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.069905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.070323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.093712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.095781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:48.104125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.104895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.105328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.105693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.106471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:48.106870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.108327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:48.119109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 315/619] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:48.123428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:48.124398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:48.125396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:48.125396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:48.125396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:48.125396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 305/620] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:48.130718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.131715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.171697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:48.204176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:48.353953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587863bd-0398-459f-a986-bdcacdf19bff None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.366024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145f84f-41d7-4da2-b4d2-46f9b1219583 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.422376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.423013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.423293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.423559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.424206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:48.424809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.426299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:48.439856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 306/621] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:48.446639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.449759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.450761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.451253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.451743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.483889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.486112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:48.514399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:48.515427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:48.516337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:48.516337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:48.516337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:48.516337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 316/622] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:48.523386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.616284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:48.645440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:48.662434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fdd02-a49d-4486-ba4f-fec39bbab971 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.666960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35240969-3b7f-470a-83c2-42b22108981d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.699312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.700050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.700397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.700737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.701487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:48.701850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.703288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:48.717033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 317/623] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:48.726484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.727133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.727408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.727709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.730221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.738762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.744113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:48.764392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:48.765423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:48.766643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:48.767710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 307/624] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:48.779773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:48.943144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42267a-2833-4bc3-8ce5-6bf74e65e944 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.959230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f677b4f-656c-49d2-b13c-6f3b5f55ca9b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:48.985502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:48.986535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:48.987025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:48.987505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:48.996605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:48.998516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:49.000401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:49.001194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.001730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.001999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.002742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:49.003547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:49.004630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:49.013523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-de3abc10-329f-4236-a1c6-4b1a4cfa4593 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:49.014661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:49.016722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:49.017729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:49.019835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 318/625] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:15:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:49.025813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 308/626] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:49.025971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9640ce-953e-4469-9856-f67fb0c79170 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.027071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:49.027691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9640ce-953e-4469-9856-f67fb0c79170 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.027967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9640ce-953e-4469-9856-f67fb0c79170 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.029602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 319/627] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:49.033030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9640ce-953e-4469-9856-f67fb0c79170 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:49.036584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:49.186286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9640ce-953e-4469-9856-f67fb0c79170 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:49.187236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e723e-e7fe-4022-820a-19c1da161f6f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:49.229710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3402cb-14f5-4413-8061-6a0eeb2e4e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:49.230272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3402cb-14f5-4413-8061-6a0eeb2e4e0e None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.230545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3402cb-14f5-4413-8061-6a0eeb2e4e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.230814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3402cb-14f5-4413-8061-6a0eeb2e4e0e None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/ef33e7dcb56a481aa051c09d45bfc83d/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.241859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:49.242856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.243962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.244812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.246957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:49.248103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:49.249571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3402cb-14f5-4413-8061-6a0eeb2e4e0e None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=ef33e7dcb56a481aa051c09d45bfc83d, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:49.250994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3402cb-14f5-4413-8061-6a0eeb2e4e0e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:49.251731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:49.269437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 320/628] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:15:49 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/ef33e7dcb56a481aa051c09d45bfc83d/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:15:49.272901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.275525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.276021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.287789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 309/629] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:15:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:49.295790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:49.299286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:49.300736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:49.374879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:49.419947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 321/630] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:49.440988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:49.500983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50fbcf-6fe6-4496-8394-41e6d8645c41 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:49.541513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:49.542118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.542393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.542658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.617071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:49.618662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:49.681898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:49.751305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 310/631] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:49.757197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:49.761686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:49.911573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746b198-694b-4042-8e5f-099ea88eb7de None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:49.953721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:49.954525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:49.954895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:49.955172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:49.969900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:49.975516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:49.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:49.977374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 311/632] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:49.990318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.030606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.035759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb52bf-1290-43ae-9900-ebd1396cb547 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.081500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.082220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.082499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.082766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.096126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:50.097645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-69ff12ee-8595-4e24-9148-42450713b710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:50.114657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:50.115730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:50.116642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:50.116642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:50.116642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:50.116642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 322/633] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:50.117555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb893-b191-43fa-a0e0-5c9b8b046069 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.117555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb893-b191-43fa-a0e0-5c9b8b046069 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.117734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb893-b191-43fa-a0e0-5c9b8b046069 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.119746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 323/634] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:50.120913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1df8c-97b4-4421-8e76-71def45cefc2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.121179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1df8c-97b4-4421-8e76-71def45cefc2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.121451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1df8c-97b4-4421-8e76-71def45cefc2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.125766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 324/635] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:50.125766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1df8c-97b4-4421-8e76-71def45cefc2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.136134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566b83-b376-4ba2-a89f-8058479ce12b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.169314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.176759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.177406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.177684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.177954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.178668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:50.178935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:50.180370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3100e-da75-419e-b3d7-0e759b464f35 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:50.194487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 312/636] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:50.194487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb68da6-5931-474c-88d5-9e3071f39b12 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.194487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb68da6-5931-474c-88d5-9e3071f39b12 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.194487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb68da6-5931-474c-88d5-9e3071f39b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.213089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2cb68da6-5931-474c-88d5-9e3071f39b12 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:50.214394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2cb68da6-5931-474c-88d5-9e3071f39b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:50.260699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1df8c-97b4-4421-8e76-71def45cefc2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.293141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 313/637] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:50.293324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.293722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.294000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.318964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:50.320194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:50.322753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.327756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.328693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.329279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.329960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.330878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:50.331365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:50.333654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:50.360587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 325/638] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:50.362655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.409033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 314/639] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:50.409033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.550369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.560822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda116ac-abfa-450c-8bbc-12b44e205b1c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.608481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1352d2-0211-40e1-98de-3f79e0ffcc11 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.617103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.617738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.618015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.618285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.630253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:50.631693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:50.648679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:50.650415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:50.651395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:50.651395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:50.651395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:50.651395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 326/640] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:50.651621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.653445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.654222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.654938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.656280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.665838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:50.667887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:50.670495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:50.687416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:50.689538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 315/641] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:50.691729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.765643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381e5e1-e273-4526-ab9d-5d6dcc281c5e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.813598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db6b4-cee4-4b0b-b77b-7c9784b50a1a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.821784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.823230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.823808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.824297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.844456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:50.848807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:50.849434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:50.849706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:50.849970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:50.850605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:50.850900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:50.852419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:50.856115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:50.857143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:50.858006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:50.861528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 327/642] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:50.861528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.872816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 316/643] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:50.874627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:50.875849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.946462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:50.980892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288dbc4-c64a-470b-89f6-e1a58575e7be None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:50.992750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e0b48-1809-442a-8ec2-eedb4107b3e3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.015683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.016282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.016558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.016825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.017537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:51.017835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.019253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.032440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 328/644] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:51.032863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.033286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.033916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.034196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.034462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.048225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:51.053967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:51.054910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:51.055894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 317/645] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:51.056622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.112602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.147682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e1715-b5a7-4baf-8a36-0b047feac064 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.166427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce36ef-b55e-494b-93c8-584ff19ed94c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.183426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.184102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.184442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.184780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.184940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.185556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:51.185928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.187327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.201101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 329/646] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:51.201101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.208734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.209455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.209798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.210132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.218043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.219506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:51.230630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:51.231744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 12:15:51.232896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:51.236393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 318/647] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:51.237178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.313984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258241-ef13-481d-8872-2666ffd17c08 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.343062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.343260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273293ec-8135-4231-b8e8-a836178b3155 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.352228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.353029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.353408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.353777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.354604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=c173404c311b43e29193004dc41480b8) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:51.354917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.356363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.373114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 330/648] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/role_assignments?user.id=c173404c311b43e29193004dc41480b8&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:51.373114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.398445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.399203 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.399778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.400098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.401410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:51.401977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.404205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.417060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 319/649] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:51.418461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.485256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.506608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b3b79-a67e-428e-ad3b-04587e6c92f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.551276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.551921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.552200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.552470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.564909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.566995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.567105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97006f-94ea-46bb-8caf-e167cf806c51 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:51.587833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:51.588824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:51.589754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:51.589754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:51.589754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:51.589754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 331/650] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:51.590632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.614396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.615161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.615429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.615689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.616413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:51.616713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.618250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.638806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 320/651] 10.222.0.85 () {56 vars in 1137 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:51.639557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.648785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.717205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56676f-e16e-49c5-8898-b192186f7e8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.758434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.758633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.760107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.760357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.760725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.763545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936afe35-263f-499a-bea9-5303cc22b65a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.777167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.779046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:51.799489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:51.800525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:51.806972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:51.806972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:51.806972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:51.806972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 332/652] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:51.806972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.815034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.816104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.816591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.817087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.817997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:51.818604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:51.820290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:51.833402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 321/653] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:51.834647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:51.917723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:51.923931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c2020-fef7-4566-b3b1-6978fbd3437f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.960142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a4421-e281-41aa-8970-123dfb32d352 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:51.960582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49dbae-8429-46a2-8f20-24debcf3fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.961227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49dbae-8429-46a2-8f20-24debcf3fb33 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.961509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49dbae-8429-46a2-8f20-24debcf3fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.961786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49dbae-8429-46a2-8f20-24debcf3fb33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:51.997735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:51.998965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:51.999240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:51.999508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.000153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:52.000444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.001842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.018656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 322/654] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:52.018656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.038869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.051391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49dbae-8429-46a2-8f20-24debcf3fb33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.053022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49dbae-8429-46a2-8f20-24debcf3fb33 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.138132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8469131-c21c-4747-bbff-db395e87b751 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.149325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.156647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 333/655] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:52.157938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c0166-52fe-4522-8afd-9ee65f011a12 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.158221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c0166-52fe-4522-8afd-9ee65f011a12 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.158491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c0166-52fe-4522-8afd-9ee65f011a12 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.163870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 334/656] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:52.163870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c0166-52fe-4522-8afd-9ee65f011a12 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.192371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606489-2035-404e-9abc-dc0207617052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.193235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606489-2035-404e-9abc-dc0207617052 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.193503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606489-2035-404e-9abc-dc0207617052 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.194125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606489-2035-404e-9abc-dc0207617052 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.203578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606489-2035-404e-9abc-dc0207617052 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.205538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606489-2035-404e-9abc-dc0207617052 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-17606489-2035-404e-9abc-dc0207617052 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:52.220254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:52.221658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 323/657] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:52.223215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606489-2035-404e-9abc-dc0207617052 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.275746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c0166-52fe-4522-8afd-9ee65f011a12 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.310275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.316137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.316791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.317101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.317369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.325309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.326746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.335314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606489-2035-404e-9abc-dc0207617052 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a24672c9-0402-4ce3-89be-185dcc43b5ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:52.339526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:52.340471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 335/658] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:52.342186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.342942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.342942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.363426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:52.364608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:52.377605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.378328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.378676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.379127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.390702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.392475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:52.411035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:52.412085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:52.418928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:52.418928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:52.418928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:52.418928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 324/659] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:52.418928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.422363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.443951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 336/660] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:52.444696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.549701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe15cb8-7a30-4e1f-b50b-b844bc204b1c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.580042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.590701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.591318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.591591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.591887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.592675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:52.592975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.593876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02343422-a2e5-451d-b4d5-843f57a2f9c0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.594676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.614836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 325/661] 10.222.0.85 () {56 vars in 1137 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:52.615689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.636228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.637520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.638023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.638500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.639460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:52.640368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.642293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.662803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 337/662] 10.222.0.85 () {56 vars in 1139 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:52.664586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.718977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.727982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5af2c2-6197-4891-82aa-fc68d0ffd791 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.765135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.765876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.766280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.766717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.768409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:52.769198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.771316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.780371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb6e3a-4b2e-4f46-aa2b-2589fb0d3ce7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.793458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 326/663] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:52.793458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:52.829433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.830062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.830363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.830680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.848417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:52.907494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64aa21-1f88-4251-b339-d8ff48640852 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:52.919791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:52.919791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:52.944967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:52.945599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:52.945874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:52.946145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:52.971113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.031857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 338/664] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:53.031857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.048627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.050120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.106111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.151817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217986b5-aa5a-4726-b545-e49172efd912 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.172603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 327/665] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:53.172603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.195197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.195825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.196102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.196381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.208322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:53.213791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:53.214903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:15:53.216638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:53.219770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 339/666] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:53.220660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.224959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.285700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d88b7-ee3a-4135-ae91-e7b790e509f8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.321608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.322842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.323115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.323372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.331156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.332631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.344628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b14e3f-59b8-4935-8cdb-4b629ca34e17 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:53.345512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:53.346927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:53.348201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:53.351746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 328/667] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:53.352497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.363505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.391713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.392441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.392770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.393142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.405675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.407605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:53.424949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:53.426302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:53.428891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:53.428891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:53.428891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:53.428891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 340/668] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:53.428891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.473372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4041079-f8cb-4a7b-9d89-6437ba9fd74a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.492351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.513984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.514617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.514894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.515164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.530504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.532452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.544206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947ef7e-2c40-431a-854b-ca358210ef0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:53.556103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:53.557638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:53.558535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:53.558535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:53.558535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:53.558535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 329/669] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:53.559187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.585312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.585925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.586208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.586475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.587141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:53.587441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.588838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.602465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 341/670] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:53.602465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.640450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.692443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f8fc-e24a-4329-a716-de70150ae68d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.733400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.734001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.734277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.734623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.735008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aaca9-434d-45ab-9e7c-db20a51d6ceb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.735238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:53.735542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.736970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.750058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 330/671] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:53.750818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.753320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.773173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.773780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.774053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.774327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.774967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:53.775282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.776633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.788735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 342/672] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:53.789581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.867175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409c90-6d96-4439-9ef1-2c5823b49b76 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.890870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:53.903608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee267d0-09b0-4982-bc09-f69f35fe2740 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:53.906190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.907349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.907349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.907349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.908016 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:53.908382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.909865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09d58b-0bbb-4df9-8fc1-d63a9cf68952 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.922666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 331/673] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:53.923951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924faa03-135e-431b-a76f-0ca1c93eb0fe None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.924485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924faa03-135e-431b-a76f-0ca1c93eb0fe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.924485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924faa03-135e-431b-a76f-0ca1c93eb0fe None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.925904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 332/674] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:53.928214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924faa03-135e-431b-a76f-0ca1c93eb0fe None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:53.943007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:53.943652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:53.943931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:53.944199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:53.955565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:53.957053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:53.972878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:53.974134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:53.975246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:53.975246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:53.975246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:53.977171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 343/675] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:53.978880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.011351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.037692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924faa03-135e-431b-a76f-0ca1c93eb0fe None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.072265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.072924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.073152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.073418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.074138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=ce6a8adf15e3498d95442b3ad0ef3864) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:54.074440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.075880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.081691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 333/676] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/role_assignments?user.id=ce6a8adf15e3498d95442b3ad0ef3864&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:54.082478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.082699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39bb9-5104-4835-ae11-c9c5a167cf4b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.123810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.124453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.124736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.125340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.134054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.136337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-76275bee-620e-446c-a78a-f0026587f4c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:54.150197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:54.151396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:54.152746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 344/677] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:54.153357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.153357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.153792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.161708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.175796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:54.177536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:54.229365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a0615-8ae7-4834-83e6-c8b63d54c780 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.262510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 345/678] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:54.263292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.269932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.270658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.271002 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.271343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.280491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.282462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.292178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:54.296287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:54.297359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:54.299634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:54.302463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 334/679] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:54.303539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.408412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f622385-c89e-45be-8029-1a073decf2a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.428736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b050dd-61be-4b77-83d0-ffaac0866f6a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.431489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.448350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.448968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.449271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.449538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.450186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:54.450784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.452251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.467752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 346/680] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:54.467752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.473387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.474485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.474485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.474704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.475321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:54.475632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.477104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.490731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 335/681] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:54.496835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.565123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.584471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffce4-7b11-4c23-b873-a54c60cd439a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.617803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafb420-c52f-49dd-8d4a-e4184a66c584 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.620077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.620712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.621068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.621345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.622050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:54.622355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.623779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.637441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 347/682] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:54.638273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.661639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.663155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.663867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.664314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.674006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.676681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:54.691927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:54.693152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:15:54.694326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:54.696937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 336/683] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:54.698543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.731673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.756882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df343ac-8d02-460c-a666-0fd4bc205674 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.795068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.795776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.796012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.796386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.797341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=b840491b8d05489c8475e1125dbc0b5b) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:54.798089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:54.799618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:54.808349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 348/684] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/role_assignments?user.id=b840491b8d05489c8475e1125dbc0b5b&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:54.809636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.817825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:54.823504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac6e9d-dbde-4b28-95d4-48f9501e0a7c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:54.867145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:54.868122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:54.868491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:54.868855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:54.882091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:15:54.889862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:54.891079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:15:54.892161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:54.898936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 337/685] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:54.898936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:54.952461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b6149-b73b-4c48-b4e1-23e530ae50f8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.002163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.003444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.004392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.004882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.006563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=e102dacae430417aa150ddd53cf34f31) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:55.007085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.009582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.012075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:55.018396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 349/686] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/role_assignments?user.id=e102dacae430417aa150ddd53cf34f31&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:55.019613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.039648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2e38-b978-4954-97e0-9b4347cd5cfe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.099847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.100472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.102076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.102076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.102371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:55.102827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.104891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.117441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:55.132175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 338/687] 10.222.0.85 () {56 vars in 1135 bytes} [Wed Nov 2 12:15:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:55.140600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.165249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed52001-6e00-4ff9-b3ca-b9dc10425fc4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.209469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.210482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.210970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.211496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.212239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:55.212611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.214144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.230115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 350/688] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:55.231362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.274019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb87c3-ba2e-45a4-a2d6-6447e6306037 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.324412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.325505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.326343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.326821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.347850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9c385-da67-45ec-b655-ab5451dccd22 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.350124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:55.381442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:55.388797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.389997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.390403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.390794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.402632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.404642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.419280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.421045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:55.422538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:55.423467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:55.424356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:55.424356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:55.424356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:55.427141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 351/689] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:55.430452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.553667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 339/690] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 12:15:55.560181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.585573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c64e6-e79d-4cbe-830e-8b0bb25dd4a3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.615779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:55.651625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.652232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.652506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.652975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.655095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:55.655413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.656772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.681119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 352/691] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:55.684168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.727084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215005-a69f-4b11-95d7-55791328af2a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.757375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:55.795509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.796154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.796482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.796750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.815464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.817021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.845180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82366-3ffc-4f11-abfd-7f3beef3b661 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:55.846175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:55.847178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:55.848075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:55.848075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:55.848075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:55.848075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 340/692] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:55.854503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:55.922261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:55.924120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:55.924403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:55.924675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:55.946462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:55.947964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:15:55.965850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:55.966861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:55.967806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 353/693] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:55.973183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.073185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:56.078880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cf8f5-c5cd-4f66-8817-d7ec2741c08a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.128664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0305c9-319c-4009-b138-e9b02b81e614 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.141427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:56.153923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.154456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.154670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.154922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.155574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:56.155873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.157270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.170103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.171492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.171971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.172492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.173456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:56.175616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.178196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.188058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 341/694] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:56.189331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.212356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 354/695] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:15:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:56.226087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.361124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8988-715e-42f5-8cf7-049ce72dea45 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.394423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.395060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.395336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.395608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.403298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9e04a-fa9c-4b78-bde0-3cc983cc6013 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.404573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.406275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:56.419941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:56.420934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:56.421889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 342/696] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:56.427688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.447302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:56.448223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea4ed2-4bf2-4346-b80a-41aab14a3ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.449201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4ed2-4bf2-4346-b80a-41aab14a3ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.449687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4ed2-4bf2-4346-b80a-41aab14a3ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.450189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4ed2-4bf2-4346-b80a-41aab14a3ca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.483034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:56.528133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea4ed2-4bf2-4346-b80a-41aab14a3ca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.528133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea4ed2-4bf2-4346-b80a-41aab14a3ca4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.592867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb70e4-64f3-4c8e-bc48-b152cf42c72e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.636283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.636969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.637452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.637728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.638597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:56.638936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 355/697] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:15:56.639074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.640220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e77a-04b4-4eee-b8fa-8d1de35fa5da None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.640506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e77a-04b4-4eee-b8fa-8d1de35fa5da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.640695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839e868-7bd2-4e5f-a21c-47275cc4f410 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.640791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e77a-04b4-4eee-b8fa-8d1de35fa5da None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.642310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 356/698] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:56.645364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e77a-04b4-4eee-b8fa-8d1de35fa5da None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.660917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 343/699] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:56.662578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.662882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.663150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.686677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:56.688108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:56.713164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:56.767819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e77a-04b4-4eee-b8fa-8d1de35fa5da None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.786385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 344/700] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:56.787380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.790765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:56.858288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.858928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.859203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.859470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.890441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.891889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.911671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360356b-3c9c-4b97-b2df-edfe5cb50b51 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:56.914497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:56.917163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:56.919252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:56.919252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:56.919252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:56.919252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 357/701] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:15:56.925298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:56.951253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:56.951891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:56.952234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:56.952504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:56.953328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=ef33e7dcb56a481aa051c09d45bfc83d) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:56.953633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:56.956291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:56.961865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 345/702] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:56 2022] GET /identity/v3/role_assignments?user.id=ef33e7dcb56a481aa051c09d45bfc83d&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:57.071163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047989df-1416-4f19-b6e1-54379c23ee82 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.089817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:57.114932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.115558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.115833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.116330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.117042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:57.117324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.118712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.134212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 358/703] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:57.141473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:57.173471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:57.294796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:57.336710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f569b-8d88-4f15-9a2d-97fe197df885 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.343522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302580f1-5ea5-49d2-9a83-e5895552be46 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.382778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.383424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.383700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.383971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.386282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.386911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.387187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.387456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.395230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.396811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.400832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.402705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:15:57.409407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:57.410335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:57.411656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 346/704] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:57.418049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:15:57.423786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:57.424774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:15:57.425729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:15:57.425729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:15:57.425729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:57.427848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 359/705] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:57.437230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:57.478552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:57.576183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5628-3d17-4ac7-865f-2cc4c9ebbd20 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.594357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e931ed-e567-48ba-a1f7-2b0c512e3383 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.615085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:57.628491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.630157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.630811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.631391 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.632394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:57.632923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.634917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.636323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.636929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.637246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.637511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.638146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:57.638444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.640134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.653114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 347/706] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:57.659866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 360/707] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:57.661252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:57.661472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:57.758304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:57.779262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d0f33-a2c2-4fec-9ebd-e910c58541c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.816747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f298d-1565-47e9-acc8-83eac124f2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.817169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94afeb-d2e4-450b-bd91-863416e428af None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:57.817384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f298d-1565-47e9-acc8-83eac124f2d8 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.817656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f298d-1565-47e9-acc8-83eac124f2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.817920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f298d-1565-47e9-acc8-83eac124f2d8 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.818566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f298d-1565-47e9-acc8-83eac124f2d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.819974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f298d-1565-47e9-acc8-83eac124f2d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.849135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 361/708] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:15:57 2022] POST /identity/v3/users => generated 254 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:15:57.849135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87276b-6018-4706-92eb-7984b2832ccf None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.849135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87276b-6018-4706-92eb-7984b2832ccf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.849135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87276b-6018-4706-92eb-7984b2832ccf None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.849135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 362/709] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:57.854452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335be5-941a-4506-96ec-49b151162996 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.854912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335be5-941a-4506-96ec-49b151162996 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.855189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335be5-941a-4506-96ec-49b151162996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.870238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:57.871321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.871852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.872340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.873452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=c173404c311b43e29193004dc41480b8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:57.874022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:57.875834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-53335be5-941a-4506-96ec-49b151162996 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:57.876986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f3604-8162-41ea-93f5-c7f09ce93104 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:57.877162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53335be5-941a-4506-96ec-49b151162996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:57.890809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 348/710] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:15:57 2022] GET /identity/v3/role_assignments?user.id=c173404c311b43e29193004dc41480b8&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:57.892735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dceaa-2f17-43fb-b356-448e889fe371 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.893345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dceaa-2f17-43fb-b356-448e889fe371 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.893843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dceaa-2f17-43fb-b356-448e889fe371 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.895779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 349/711] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:57.913361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:57.914041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:57.914537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:57.958648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:57.959951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:15:57.988559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 363/712] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:15:58.059913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:58.091379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 350/713] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:15:58.347203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335be5-941a-4506-96ec-49b151162996 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:58.370946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:58.430872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:58.474285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335be5-941a-4506-96ec-49b151162996 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:58.509486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:58.510120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:58.510428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:58.510786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] PATH_INFO: `/v3/services/network` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:58.511270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:58.512617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:58.536323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:58.537501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:15:58.538361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:15:58.542498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 364/714] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:15:58 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:58.547982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:58.588247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95bcf9-792f-4510-8f91-c16046dabd96 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:58.638043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:58.638607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:58.638880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:58.639148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:58.639732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:58.641127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:15:58.661308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:15:58.662363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:15:58.663639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:15:58.664382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 351/715] 10.222.0.85 () {56 vars in 1129 bytes} [Wed Nov 2 12:15:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:15:58.672627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:58.731176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82d925-3b8b-488c-b5e5-ed4bc27b8e11 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:58.809806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:58.811691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:58.814506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:58.814985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:58.815471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:58.816665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:58.817253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:58.819909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:58.831283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:58.846566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 365/716] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:15:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:58.850730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:58.946477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592c78a-4980-4549-aa43-76a318141f2a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:58.994738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:58.995358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:58.995728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:58.996002 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:58.996634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:58.996931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:58.998591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:59.004314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 352/717] 10.222.0.85 () {56 vars in 1124 bytes} [Wed Nov 2 12:15:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:59.012106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:15:59.082030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84f73c-690d-4ca6-bc7e-8b9bb6400387 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:59.128549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:59.129196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:59.129470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:59.129736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:59.130366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:59.130661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:59.132020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0693b5-99b4-4ca6-8f3d-eba7720bb48e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:59.139176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 366/718] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:15:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:15:59.247619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8857bc-9d42-4419-896a-570d4178a54f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:15:59.269830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:15:59.309804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:15:59.310496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:59.310815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:59.311133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:59.311856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:15:59.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:15:59.313786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4ef8b-162c-4ec5-85f2-2d1412bdf561 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:15:59.321359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 353/719] 10.222.0.85 () {56 vars in 1124 bytes} [Wed Nov 2 12:15:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:15:59.888143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad62416-5f79-46d1-aa90-a131c302b990 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:59.888674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad62416-5f79-46d1-aa90-a131c302b990 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:59.888973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad62416-5f79-46d1-aa90-a131c302b990 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:59.890637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 367/720] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:15:59.896244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b31d15-1713-42e2-9c56-e2c68eaeda94 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:15:59.896652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b31d15-1713-42e2-9c56-e2c68eaeda94 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:15:59.896847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b31d15-1713-42e2-9c56-e2c68eaeda94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:15:59.922045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a0b31d15-1713-42e2-9c56-e2c68eaeda94 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:15:59.923266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0b31d15-1713-42e2-9c56-e2c68eaeda94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:00.017216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 354/721] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:16:00.513473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad62416-5f79-46d1-aa90-a131c302b990 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:00.609501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:00.664348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad62416-5f79-46d1-aa90-a131c302b990 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:00.705402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:00.706039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:00.706313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:00.706580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:00.716730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80267e72-9534-4b87-8a43-41bb4d613949 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:00.717037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80267e72-9534-4b87-8a43-41bb4d613949 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:00.717472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80267e72-9534-4b87-8a43-41bb4d613949 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:00.719174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 355/722] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:00.724375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:00.725380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eeb68-d26d-4dff-9274-8e452577791f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:00.725656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eeb68-d26d-4dff-9274-8e452577791f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:00.725921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eeb68-d26d-4dff-9274-8e452577791f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:00.729892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:00.730824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:00.731671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 368/723] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:00.738632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:00.763814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-795eeb68-d26d-4dff-9274-8e452577791f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:00.765013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-795eeb68-d26d-4dff-9274-8e452577791f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:00.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308968f-9473-437f-b017-5c3ac48c77bf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:00.893553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:00.922348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:00.923366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:00.923847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:00.924358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:00.925326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:00.925848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:00.927495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:00.945782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 369/724] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:00.952188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:00.956133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 356/725] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:16:01.041495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b2253-a815-42a8-8c91-8219570003a3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.042130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b2253-a815-42a8-8c91-8219570003a3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.042600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b2253-a815-42a8-8c91-8219570003a3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.044145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 357/726] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:01.049364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.050053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.050565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.072620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:01.075601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:01.110159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372441-f40e-4eea-bd27-6c5699587b9c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.131432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:01.185222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.185928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.186210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.186482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.206887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:01.208573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:01.214175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 358/727] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 12:16:01.238096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:01.238532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:01.239503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:01.240415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:01.240415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:01.240415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:01.242599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 370/728] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:01.248779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.319191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:01.452485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4c889-b09e-42b5-a37b-0f428bf3e142 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.460671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c010a-f555-49d6-8253-c301dad9b7aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.516624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.516841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.517402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.517856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.518136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.518397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.518857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.519334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.520168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:01.520658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:01.522222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:01.531127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:01.535179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 371/729] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:01.536274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:01.537292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:01.538439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:16:01.539495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:01.542674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 359/730] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:01.543713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.589828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:01.649077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:01.655680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20332e0b-46a1-4dbc-93f4-ad2044752d85 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.678877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde923-6394-4199-95be-665aa6a73432 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.693788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.694679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.695133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.695550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.707321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:01.712771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:01.714414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:01.715256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:01.717888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 372/731] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:01.718942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.728504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.729572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.730340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.730611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.741654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:01.743849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:01.761439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:01.762459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:16:01.763427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:01.766554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 360/732] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:01.768058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.838666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb683f-a65a-47f9-85b6-fd11025848ca None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.870174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:01.876538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.877619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.877864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.878141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.879070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:01.879376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:01.880875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:01.895050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fcecf-327a-4a72-afeb-bc19de2bd7a4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:01.895241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 373/733] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:01.895241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:01.913012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:01.934305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:01.934908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:01.935184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:01.935451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:01.936111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:01.936459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:01.937855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:01.946625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 361/734] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:01.947434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.007508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2e25c-53fb-4103-a86d-1c41985dda16 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.045175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.045779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.046053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.046318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.047039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:02.047339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.048740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24166dda-37f0-4e78-a962-ff836ecca07f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.054068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c9826-5bf4-4262-9d17-5bb669211626 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.071554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 374/735] 10.222.0.85 () {56 vars in 1135 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:02.071554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db848-6a9a-44dc-afb0-a744254ad020 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.071554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db848-6a9a-44dc-afb0-a744254ad020 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.071554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db848-6a9a-44dc-afb0-a744254ad020 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.071554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 375/736] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:02.071554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db848-6a9a-44dc-afb0-a744254ad020 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.103032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.104071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.104559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.104984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.116471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.117987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.124932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:02.133543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:02.134716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:02.136375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:02.136375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:02.136375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:02.138042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 362/737] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:02.139001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.169349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.198092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db848-6a9a-44dc-afb0-a744254ad020 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.232877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.233548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.233856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.234126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.246035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.247474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.249953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66db4-f8bf-4fad-bca5-a5c1efd45172 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-eade9cb0-0f80-4666-aa5c-06f5fbde1753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:02.263853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:02.265463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:02.266781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:02.266781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:02.266781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:02.266781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 376/738] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:02.267353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.267633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.267899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.285177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:02.286353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:02.287099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a29662-2242-4651-948e-f7a2acdee03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.287700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a29662-2242-4651-948e-f7a2acdee03e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.287976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a29662-2242-4651-948e-f7a2acdee03e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.288249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a29662-2242-4651-948e-f7a2acdee03e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.351165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 377/739] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:16:02.352005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.355456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.359133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a29662-2242-4651-948e-f7a2acdee03e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.360924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a29662-2242-4651-948e-f7a2acdee03e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.423503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.469691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26134f1-944d-44c1-98eb-2513c3107485 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.475034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 363/740] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 12:16:02.475034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe31e2a-50d9-469d-bfdc-0bb0c494ee0c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.475034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe31e2a-50d9-469d-bfdc-0bb0c494ee0c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.475034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe31e2a-50d9-469d-bfdc-0bb0c494ee0c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.475034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 364/741] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:02.475034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe31e2a-50d9-469d-bfdc-0bb0c494ee0c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.514164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.514776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.515054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.515326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.515977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:02.516277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.517669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.529925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 378/742] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:02.530574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.583763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.590753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe31e2a-50d9-469d-bfdc-0bb0c494ee0c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.629569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.630174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.630455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.630720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.631367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:02.631670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.633069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6751-64a1-4682-b469-64d2ddee2c82 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.642514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8a833-15b8-4f75-a3fa-118e75a4e23a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.644338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 365/743] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:02.645575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.645853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.646124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.660555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.664063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:02.665392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:02.678810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.679506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.679783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.680081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.690805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.692240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:02.706256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:02.707214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:02.708126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:02.708126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:02.708126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:02.708126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 379/744] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:02.708417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.733849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 366/745] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 12:16:02.734682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.825855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:02.826058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ab25c-6638-486d-b3ca-8eac815cf864 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.838319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589e12f-d379-436c-a221-7986b95bb92b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:02.863530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.864149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.864421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.864686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.872235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.873658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:02.873783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.874257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:02.874535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:02.874806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:02.875454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:02.876860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:02.885163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:02.887217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 380/746] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:02.888190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.899262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 367/747] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:02 2022] POST /identity/v3/services => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:02.902622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:02.938801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.026382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb93c0-b589-4371-b7de-7adc5eb6f283 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.049650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e17b-0bea-43a0-9b8f-1452861ed944 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.068138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.069042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.069633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.069926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.080951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.082468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.090376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.090991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.091263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.091530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.092176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:03.092501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.094647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:03.095010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:03.095942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:03.096898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 381/748] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:03.097544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.110163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 368/749] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:03.114210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.114633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.202241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.214360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb83cd-f044-4035-b796-cd8ae7e536f9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.227342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2c6cd-a4f9-4dde-a924-4dec6fe2f705 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.252144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.253117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.253544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.253824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.254809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:03.255121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.256552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.265476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.266112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.266384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.266645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.267323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.268742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.289212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 369/750] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:03 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:03.289212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.376640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.397028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 382/751] 10.222.0.85 () {56 vars in 1149 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:03.398028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.449978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52ed86-03ec-42bb-82e8-bfeff3d71dec None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.512437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.513102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.513384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.514560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.514560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:03.514741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.516087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.527704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d076336-66a5-4e53-a006-c0194ad57bfb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.535378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 370/752] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:03.536225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.570090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.570719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.571004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.571251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.579693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.581263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:03.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:03.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:03.599676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 383/753] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:03.600391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.613884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.700288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebfe5e-6819-4e8a-b2d5-e2564229872e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.723963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.736414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837527-dd58-4aca-ad06-eba8ec665f31 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.752211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.753461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.754683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.755154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.779044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:03.779701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:03.780000 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:03.780264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:03.795818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.797979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:03.817751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:03.818890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:03.819797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:03.819797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:03.819797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:03.823793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 384/754] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:03.825093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:03.847677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:03.850009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:03.913446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:03.960254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a87701-9dd3-416d-94a7-f53452be1232 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:03.998363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 371/755] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:04.000291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.008302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.009071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.009464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.009829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.010615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:04.011053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.013577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.028882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:04.031753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 385/756] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:04.033029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.164616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e0b8-b09f-4449-b5f8-3dcd7496103d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.178815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fc219-00c6-40cf-b4c4-5fbfa1aa3f12 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.202925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.203682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.204054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.204422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.205789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:04.205789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.207175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.213221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:04.226883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 372/757] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:04.228007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.229890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.231220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.231639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.232073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.243804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.245606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:04.261234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:04.262232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:04.263139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:04.263139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:04.263139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:04.263139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 386/758] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:04.264399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.342435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:04.366739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e6889-d344-4de9-92d7-51b418b01481 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.400484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11a31-e85c-4d4a-957f-367975c388f9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.404918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.405223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.405436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.405705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.406441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=ef33e7dcb56a481aa051c09d45bfc83d) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:04.406752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.408209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.414907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 373/759] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:04 2022] GET /identity/v3/role_assignments?user.id=ef33e7dcb56a481aa051c09d45bfc83d&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:04.415930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.446422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.446809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.447190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.447461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.456054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.457672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:04.474368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:04.475331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:04.476258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 387/760] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:04.482229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.519028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:04.618482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40fa1a-4850-4adb-8716-c96505d5f3ee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.631676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acdad8-57ab-4d66-8f3f-9efbcf6d4edc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.664939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.666487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.666766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.667037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.667913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:04.668217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.669687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.687745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 388/761] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:04.693962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.704418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.707306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.707843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.708351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.709657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:04.710140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.713736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.737922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 374/762] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:04.742195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:04.746285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:04.885610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:04.901213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8ac3b-197c-4e39-8ba2-0659a7e28b05 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.903307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14ee8-9d73-49eb-b137-b3b6bb27bf9d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:04.971368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.973820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.974884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.975674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/c173404c311b43e29193004dc41480b8/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.980288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=c173404c311b43e29193004dc41480b8, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.982872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:04.983698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:04.984532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:04.987422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:04.987915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:04.997887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:04.999616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:05.007582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 389/763] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:16:04 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/c173404c311b43e29193004dc41480b8/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:05.020733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:05.021765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:05.022657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:05.024877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 375/764] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:05.033742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:05.172663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72291b9-eb9f-4ad5-98e1-e788be2d8b8d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:05.221832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:05.222738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:05.223189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:05.223627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:05.224499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:05.224966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:05.226611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd61a5-429f-41cb-a046-a5fe62c2f1f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:05.244992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 390/765] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:05.251841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:05.336974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:05.350785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db114a1-54d1-4d2c-bb3f-a334cba56b83 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:05.351420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db114a1-54d1-4d2c-bb3f-a334cba56b83 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:05.351895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db114a1-54d1-4d2c-bb3f-a334cba56b83 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:05.353930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 391/766] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:05.360260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b3a5-558c-4681-ad0c-c4874d2adbf5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:05.360896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b3a5-558c-4681-ad0c-c4874d2adbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:05.361462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b3a5-558c-4681-ad0c-c4874d2adbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:05.387746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-df32b3a5-558c-4681-ad0c-c4874d2adbf5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:05.388960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-df32b3a5-558c-4681-ad0c-c4874d2adbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:05.403506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:05.412415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fbe56-7b68-4d8f-a464-9f4d059f3500 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:05.451786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:05.452402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:05.452677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:05.452946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:05.453724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=b5445ced373148aeb5741e1e11e897cb) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:05.454031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:05.455713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870bf35-6957-46cb-bbbf-04b928a4e678 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:05.469313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 376/767] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:05 2022] GET /identity/v3/role_assignments?user.id=b5445ced373148aeb5741e1e11e897cb&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:05.482347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 392/768] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:05.887353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79034a-c60b-4d7c-921f-b7a6015c8b25 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:05.887765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79034a-c60b-4d7c-921f-b7a6015c8b25 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:05.888150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79034a-c60b-4d7c-921f-b7a6015c8b25 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:05.889902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 377/769] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:05.895156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:05.895441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:05.895712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:05.919802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:05.921021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:05.977754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79034a-c60b-4d7c-921f-b7a6015c8b25 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:06.000501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 393/770] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:06.116775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79034a-c60b-4d7c-921f-b7a6015c8b25 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:06.145759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:06.155689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:06.156326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:06.156606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:06.156875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:06.157494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:06.158931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:06.168367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:06.169398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:16:06.170248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:06.175654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 378/771] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:05 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:06.175788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:06.259005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:06.299552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be5e43-68d6-4a2e-805f-132624385821 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:06.303280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:06.340110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:06.340726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:06.341016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:06.341287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:06.342404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:06.342715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:06.344242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:06.349393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 394/772] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:06.354904 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:06.512422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b878ce-e486-48ff-8a9d-4907f7b294c3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:06.546717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7716c-d504-4998-8122-a8c334a7c308 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:06.597103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:06.597103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:06.597103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:06.597103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:06.597519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:06.597519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:06.609700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:06.610726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:16:06.611575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:06.612319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 379/773] 10.222.0.85 () {56 vars in 1129 bytes} [Wed Nov 2 12:16:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:06.620106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:06.625976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:06.626656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:06.626937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:06.627208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:06.627847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:06.628148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:06.629795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:06.633594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:06.643322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 395/774] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:06.652224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:06.771372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:06.929060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd520a-2936-45b6-9cbf-6b4b9989cfca None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:06.940490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d410fa-2075-4fad-a085-a61a0da2aac7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:07.031356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:07.031879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.032141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.032414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.034597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:07.034798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] RBAC: Adding query filter params (interface=public, service_id=1117eba53d22497192310560f92ecd41, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:07.041027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.041160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:07.041253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.041253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.041253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:07.041460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:07.041531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:07.041531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a272da-385a-4b2f-8310-7726ebc699bd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:07.047893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 380/775] 10.222.0.85 () {56 vars in 1124 bytes} [Wed Nov 2 12:16:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:07.052902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1af38-3b91-4258-a38b-ae2a2d5a7887 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.053228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1af38-3b91-4258-a38b-ae2a2d5a7887 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.053505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1af38-3b91-4258-a38b-ae2a2d5a7887 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.058277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 396/776] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:06 2022] GET /identity/v3/endpoints?service_id=1117eba53d22497192310560f92ecd41&interface=public®ion_id=RegionOne => generated 194 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:07.059626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:07.066440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 381/777] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:07 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:07.066440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1af38-3b91-4258-a38b-ae2a2d5a7887 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:07.199228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:07.210831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1af38-3b91-4258-a38b-ae2a2d5a7887 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:07.230179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:07.245231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2e160-b755-434c-a8bb-6722c8132d10 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:07.250876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9054f-74fd-4156-b253-332f1c3dc21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:07.251497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9054f-74fd-4156-b253-332f1c3dc21b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.251775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9054f-74fd-4156-b253-332f1c3dc21b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.252046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9054f-74fd-4156-b253-332f1c3dc21b None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.252663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9054f-74fd-4156-b253-332f1c3dc21b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:07.254153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9054f-74fd-4156-b253-332f1c3dc21b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:07.260395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 382/778] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:07 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:07.261773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.262061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.262337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.284590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:07.286294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:07.330507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:07.331591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.332064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.332545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.333779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:07.334289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:07.336989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:07.359858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 397/779] 10.222.0.85 () {56 vars in 1124 bytes} [Wed Nov 2 12:16:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:07.365334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:07.375574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 383/780] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:07.548808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd691c9-52a7-40b6-935d-e697dfe2e10e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:07.581958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:07.592901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:07.593548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.593823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.594086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.594795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] RBAC: Adding query filter params (interface=public, service_id=fd627ff8188a467cb517996aa5bf9cdf, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:07.595095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:07.596472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:07.602911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 398/781] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:07 2022] GET /identity/v3/endpoints?service_id=fd627ff8188a467cb517996aa5bf9cdf&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:07.608776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:07.728692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae77bfe-243c-4975-8627-51448c58d3a1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:07.747733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:07.780339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24468581-79aa-49ef-bbac-a23868122891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:07.781070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24468581-79aa-49ef-bbac-a23868122891 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.781347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24468581-79aa-49ef-bbac-a23868122891 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.781612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24468581-79aa-49ef-bbac-a23868122891 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.782234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24468581-79aa-49ef-bbac-a23868122891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:07.783676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24468581-79aa-49ef-bbac-a23868122891 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:07.794255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 384/782] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:07 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:07.846019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:07.959091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890667c0-ce21-4a74-ac61-6e5e1f63b4d0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:07.986305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ab6b-e613-467a-ab89-2748f2ef9cf2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.986582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ab6b-e613-467a-ab89-2748f2ef9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.986857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ab6b-e613-467a-ab89-2748f2ef9cf2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.988545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 385/783] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:07.993810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75cf8-e0fa-4b54-a66a-fd9dc39cf35f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:07.994083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75cf8-e0fa-4b54-a66a-fd9dc39cf35f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:07.994348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75cf8-e0fa-4b54-a66a-fd9dc39cf35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:07.999757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.000731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.001235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.001805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.016267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:08.021663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:08.022683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:08.023545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:08.032275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 399/784] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:08.034423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.039292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-55b75cf8-e0fa-4b54-a66a-fd9dc39cf35f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:08.040887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-55b75cf8-e0fa-4b54-a66a-fd9dc39cf35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:08.101520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:08.158240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a79d0-a478-4cf8-b22b-7ed7d540ac53 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:08.173280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 386/785] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:08.174994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb4557-737f-4e41-9d85-6b19d369e638 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.175507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb4557-737f-4e41-9d85-6b19d369e638 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.175830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb4557-737f-4e41-9d85-6b19d369e638 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.177338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 387/786] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:08.182807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.183471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.183956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.203403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:08.203627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.204031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.204477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.204905 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.205406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:08.205945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:08.206397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:08.208058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:08.221708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 400/787] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:08.224274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.345485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 388/788] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:08.388342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c4e9-2891-4c18-b6ea-4a3cc4a9b8f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:08.403285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:08.426197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.426832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.427105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.427367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.440538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:08.442108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:08.458524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:08.459559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:08.460671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:08.460671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:08.460671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:08.460671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 401/789] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:08.467022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.484773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.506519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:08.615237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d234896-f158-4341-93c3-f44faa0fd10a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:08.623463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c510833-6f58-4c0f-a126-f9f5472bbd3f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:08.660580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.661139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.661417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.661685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.662325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:08.662627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:08.663988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:08.676665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 389/790] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:08.682043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.682829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.683105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.683887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.684439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.699618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:08.704954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:08.705957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:08.706792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 402/791] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:08.707563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.737120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:08.812821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:08.822464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0659f7-7783-4048-a9be-92f0f6aefeb8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:08.828440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81420fbf-01ca-4068-8237-a0a00d90b6f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:08.867517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.868148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.868404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.868670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.872000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:08.872885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:08.873377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:08.874111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:08.880116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:08.884093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:08.885360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:08.886274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:08.887644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 403/792] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:08.888336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:08.893068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:08.906231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:08.907229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:16:08.908174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:08.910940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 390/793] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:08.911976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.015174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.029480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076a920-ff04-44ff-810e-066a9c880148 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.037540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87a8f9-424e-4e1a-bd25-e4de4c6b4283 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.072397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.073043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.073319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.073594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.074884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:09.075191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.075906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.076626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.077023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.077495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.077952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.078906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:09.079500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.081189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.089084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 391/794] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:09.090955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.097126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 404/795] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:09.098320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.118074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.215888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c2e43-9581-4d0f-8f23-8466ae63b7a3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.216635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b4759-7d01-4d3d-9a19-243107910ae4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.256982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.257613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.257876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.258143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.258860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:09.259161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.260580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.266678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.267582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.268067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.268869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.280216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 392/796] 10.222.0.85 () {56 vars in 1135 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:09.281059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.282876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.285452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.300724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:09.311147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:09.312199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:09.313168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:09.313168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:09.313168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:09.317036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 405/797] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:09.317036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.387241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.418160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee83a-bee2-4ce3-9256-6a672b3376f6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.434652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022720-e405-4e78-b7ba-9cf7edfc630f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.467756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.467756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.467756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.467756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.479985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.480272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.481171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.481658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.481834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.482266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.494315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.496514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:09.501378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:09.502337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:09.503250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:09.503250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:09.503250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:09.503250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 393/798] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:09.504358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:09.518703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:09.519784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:09.520692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:09.520692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:09.520692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:09.520692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 406/799] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:09.520692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.582244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.630787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb24a7-13a3-484e-bfd0-1ed7ef828fd8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.648157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca214589-7386-4aa0-bd8d-a1fb21554f91 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.660964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.670934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.671545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.671824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.672097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.672743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:09.673064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.674436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.687907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.688517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.688793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.689090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.689738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:09.690036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.691400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.693276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 394/800] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:09.694214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.706932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 407/801] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:09.709717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.806978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada0016-21a4-4d4c-8911-54772e894f4a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.830230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74fa47-16ab-4d5a-a171-793f984e0645 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:09.848666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:09.857665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.858276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.858546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.858811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.859454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:09.860337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.862023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.877969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:09.878694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:09.878851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 395/802] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:09.879098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:09.879542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:09.879703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.888260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:09.889938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:09.903909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:09.905857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:09.908034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:09.909339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 408/803] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:09.913047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:09.943401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.000457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c0e08-00e4-4174-818f-cf547d8077ef None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.029519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7974c-ecbd-49fe-a64b-ae11d3d4f239 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.036683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.037352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.037627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.037896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.047101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.048562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:10.060469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:10.061462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 396/804] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:10.062408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.073602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.074269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.074547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.074816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.081910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.083338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-697f6d21-fbba-4791-b6f7-bc4d6e79a10e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:10.095794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:10.096685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:10.097620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 409/805] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:10.098105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bbea4-3f9e-46c0-b541-7f8886672b20 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.098478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bbea4-3f9e-46c0-b541-7f8886672b20 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.098568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bbea4-3f9e-46c0-b541-7f8886672b20 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.099946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 410/806] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:10.100490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bbea4-3f9e-46c0-b541-7f8886672b20 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.136545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.186163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a910f0-284d-471b-a875-defcffb9d09f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.196420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.224136 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bbea4-3f9e-46c0-b541-7f8886672b20 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.225916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.226365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.226641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.226914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.228058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:10.228058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.229424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.250713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 397/807] 10.222.0.85 () {56 vars in 1149 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:10.251356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.268347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.268958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.269290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.270061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.270274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:10.270554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.271972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ee5d3-661d-4ed6-bcbe-407978f1c6fd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.297436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 411/808] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:10.297436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.297436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.297436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.312391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:10.313601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:10.377262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d08ba2-9aee-410e-973d-6e70a6b9434f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.397045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 412/809] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:10.397907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.425475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.427306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.428207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.428906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.429908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.431839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:10.432418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.435658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.454095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 398/810] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:10.455138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.488807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.521673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8ccb2-e797-4057-9765-e5f592c3b01d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.561483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.562152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.562497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.562705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.571376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeaf9b-a52e-4537-b922-76ff07dba445 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.573975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.575973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-886661a5-83d4-45bf-88b4-c44f187f54ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:10.596940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:10.597995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:10.598911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:10.598911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:10.598911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:10.598911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 413/811] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:10.599594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8729a-0998-48d0-ab4b-fbc811fe49aa None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.599768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8729a-0998-48d0-ab4b-fbc811fe49aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.600078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8729a-0998-48d0-ab4b-fbc811fe49aa None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.601608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 414/812] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:10.605048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8729a-0998-48d0-ab4b-fbc811fe49aa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.619664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8d6ba-fb7e-4aef-a6aa-742fb77e6ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.620450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8d6ba-fb7e-4aef-a6aa-742fb77e6ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.620730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8d6ba-fb7e-4aef-a6aa-742fb77e6ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.621022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8d6ba-fb7e-4aef-a6aa-742fb77e6ad8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.689972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.696363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8d6ba-fb7e-4aef-a6aa-742fb77e6ad8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.697864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8d6ba-fb7e-4aef-a6aa-742fb77e6ad8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.722997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8729a-0998-48d0-ab4b-fbc811fe49aa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.726003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.758055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.758695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.759193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.759339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/ef33e7dcb56a481aa051c09d45bfc83d/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.761198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=ef33e7dcb56a481aa051c09d45bfc83d, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.763193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.776397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 415/813] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:16:10 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/ef33e7dcb56a481aa051c09d45bfc83d/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:16:10.777254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.805480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 399/814] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:10.807093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.807464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.807722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.831009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:10.832147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:10.910889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58e660-9dcf-4d06-9f11-35cc56c1bc7f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:10.925027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 400/815] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:10.925027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:10.962346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:10.969307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:10.970721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:10.971238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:10.972265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:10.973027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:10.973837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:10.975543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:10.996657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 416/816] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:10.997654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.012771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.064242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f6151-c34d-452c-b629-1b3bba9a2701 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.100537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.101164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.101441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.101759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.102366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.103726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:11.113788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:11.114862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:11.116205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 401/817] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:11.117040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.118417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e0483-872c-4523-9b31-4c2e77a384f2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.159836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.160981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.161492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.161965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.176472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.179270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:11.199098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:11.200449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:11.201619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:11.201619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:11.201619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:11.204081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 417/818] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:11.205418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.232098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139ccbf-953d-44fa-a4e4-43e76f999dbf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.260786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.267325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.278558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.279192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.281205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.281205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.287583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.288807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:11.301829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:11.302777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:11.303720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:11.307751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 402/819] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:11.308758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.341164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba02ce-7a4a-4438-b880-6cf426b8e077 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.383187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.383901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.384283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.384563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.385259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:11.385565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.387515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.392907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 418/820] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:11.393779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.446485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08287a-3abd-4aff-99a8-29ca9ddda4f9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.490229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.490909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.491533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.491533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.492068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:11.492340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.493764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.508019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 403/821] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:11.509867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.518366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d90a4f-2111-4f06-b029-cee2440de9a6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.529413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.555611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.556244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.556521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.556791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.557414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.559298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.560206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:11.569483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:11.570690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:11.572307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 419/822] 10.222.0.85 () {56 vars in 1129 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:11.576878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.626761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368833c2-6e80-449e-b466-9115f5aa22c9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.662273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.662911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.663244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.663578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.664484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:11.664694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.666318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.679401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 404/823] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:11.680251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.696547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4fb5e-f126-411d-b566-c279d9db0d4e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.734427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.735038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.735312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.735583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.736202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:11.736507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.737896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.743505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 420/824] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:11.744147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.780853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.792905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:11.796658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfb5fb-5f76-4cde-b525-9d0d34a8c5bc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.834905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.835690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.835968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.836237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.843775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.845228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.853598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd120a20-0182-417a-b971-ac230dc91260 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:11.857083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:11.858155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:11.859488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:11.863036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 405/825] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:11.863885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.893570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:11.894264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:11.894607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:11.894945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:11.895753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:11.896203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:11.897707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:11.908353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 421/826] 10.222.0.85 () {56 vars in 1124 bytes} [Wed Nov 2 12:16:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:11.908353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:11.978759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93727551-35c3-4510-b78d-ba7809ed36c5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:12.016326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:12.017022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:12.017296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:12.017599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/b5445ced373148aeb5741e1e11e897cb/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:12.021438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:12.024386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312c064-e4d4-48a1-b16f-484a51030928 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:12.033580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=b5445ced373148aeb5741e1e11e897cb, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:12.035067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:12.049152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 406/827] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:16:11 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/b5445ced373148aeb5741e1e11e897cb/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:16:12.049987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:12.068357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:12.080324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:12.081984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:12.082516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:12.083205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:12.084103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:12.087032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:12.117159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 422/828] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:12.118567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:12.182039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448faf6-b6fb-4196-81ae-458eb5d60d8e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:12.265973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:12.266519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:12.266796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:12.267064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:12.267769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:12.268070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:12.269550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:12.271437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22f33-c6eb-47e0-b9c3-4cc68380196b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:12.285503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 407/829] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:12.291059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:12.321372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:12.322060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:12.322345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:12.322617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:12.323385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:12.323694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:12.325706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:12.337411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 423/830] 10.222.0.85 () {56 vars in 1124 bytes} [Wed Nov 2 12:16:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:12.339891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:12.381241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:12.381759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:12.430099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1c3f-b79d-4a7e-8936-c92baaac607b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:12.462366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d928-241d-43cd-acbf-7f8e8b00b76a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:12.474822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:12.475443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:12.475716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:12.475976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:12.476766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=c173404c311b43e29193004dc41480b8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:12.477117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:12.478660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db81cb-79dd-4d88-bb99-ade230fd4ed5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:12.486558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 408/831] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:12 2022] GET /identity/v3/role_assignments?user.id=c173404c311b43e29193004dc41480b8&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:12.510489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224bd73-c45b-40f2-a150-4b8c27dafbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:12.511379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224bd73-c45b-40f2-a150-4b8c27dafbcf None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:12.511809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224bd73-c45b-40f2-a150-4b8c27dafbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:12.512287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224bd73-c45b-40f2-a150-4b8c27dafbcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:12.513201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224bd73-c45b-40f2-a150-4b8c27dafbcf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:12.514809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224bd73-c45b-40f2-a150-4b8c27dafbcf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:12.540631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 424/832] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:13.130917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f9c54-9f7b-4d9e-a268-6f64278f6231 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:13.131442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f9c54-9f7b-4d9e-a268-6f64278f6231 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:13.131442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f9c54-9f7b-4d9e-a268-6f64278f6231 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:13.133068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 409/833] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:13.139320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:13.139757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:13.139936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:13.158512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:13.159709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:13.227132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:13.237327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 425/834] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:13.503757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f9c54-9f7b-4d9e-a268-6f64278f6231 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:13.620011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f9c54-9f7b-4d9e-a268-6f64278f6231 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:13.650801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:13.657571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:13.658200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:13.658479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:13.658750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:13.670419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:13.675151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:13.676063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:13.676916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 410/835] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:13.682565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:13.786460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429ed62-f621-45a6-9c6d-969f4df6e859 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:13.821731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:13.822682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:13.823186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:13.823662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:13.824574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:13.825101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:13.826646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:13.835619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 426/836] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:13.841805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:13.953376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61531-024c-48ac-80a3-31e5cb1867a8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:13.990723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:13.991407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:13.991683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:13.992005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.003699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:14.005185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:14.020534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:14.021569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:14.022465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:14.022465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:14.022465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:14.022465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 411/837] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:14.027605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:14.092646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:14.141599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b830-419a-4d87-aa93-1b71a3dda526 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:14.178027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:14.178642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.178928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.179206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.179907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:14.180214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:14.181664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:14.194489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 427/838] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:14.200344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:14.265682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:14.339113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1443b4-cba5-4ebf-94ec-042b7463a57e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:14.382318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:14.382882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.383156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.383424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.393387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:14.394999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c798d627-30b1-4768-ab9f-f6319cf1ef44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:14.408442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:14.409665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:14.411138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 412/839] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:14.415504 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:14.496429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79154f0-9064-4e4c-8df7-e2ed66f1d5c4 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.497363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79154f0-9064-4e4c-8df7-e2ed66f1d5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.497622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79154f0-9064-4e4c-8df7-e2ed66f1d5c4 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.499227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 413/840] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:14.504783 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.505119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.505394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.525156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294e492-a6a6-454c-a6fc-d0e2654a6c64 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:14.525733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:14.527048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:14.546995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:14.562383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:14.563219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.563648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.563987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.564924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:14.565258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:14.566755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:14.587188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 428/841] 10.222.0.85 () {56 vars in 1135 bytes} [Wed Nov 2 12:16:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:14.593621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:14.614622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 414/842] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:14.710671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169134b4-155e-49a5-8628-97b538ae2a57 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:14.746468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb3074-a029-46c9-aac0-6bb9972e4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:14.747648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb3074-a029-46c9-aac0-6bb9972e4502 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.747930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb3074-a029-46c9-aac0-6bb9972e4502 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.748196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb3074-a029-46c9-aac0-6bb9972e4502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.799051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:14.826490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb3074-a029-46c9-aac0-6bb9972e4502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:14.828010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb3074-a029-46c9-aac0-6bb9972e4502 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:14.916935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:14.947101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 429/843] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 12:16:14.948451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a3d2-fc40-4e5c-a39b-de783e5f51a8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:14.948740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a3d2-fc40-4e5c-a39b-de783e5f51a8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:14.949036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1a3d2-fc40-4e5c-a39b-de783e5f51a8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:14.950643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 430/844] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:14.956762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a3d2-fc40-4e5c-a39b-de783e5f51a8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:14.959527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.045463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95740380-ac51-4675-ad9c-4bd0d1ef80b9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.046075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.073739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1a3d2-fc40-4e5c-a39b-de783e5f51a8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.085027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97725624-12bd-4d06-917f-8c8c19a039e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.085788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725624-12bd-4d06-917f-8c8c19a039e6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.086068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725624-12bd-4d06-917f-8c8c19a039e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.086363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97725624-12bd-4d06-917f-8c8c19a039e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.097825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97725624-12bd-4d06-917f-8c8c19a039e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-97725624-12bd-4d06-917f-8c8c19a039e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:15.102696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:15.103809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:15.104736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 415/845] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:16:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:15.105490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.105490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.105616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.116196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.116824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.117131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.117406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.124538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:15.125774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:15.128431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.129900 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5a296c15-7125-4749-8068-897acf4e8fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:15.145743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:15.147231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:15.148123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:15.148123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:15.148123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:15.148123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 431/846] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:15.148564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e2cb-6a94-4db5-afd2-7b98438cd174 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.148807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e2cb-6a94-4db5-afd2-7b98438cd174 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.149036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646e2cb-6a94-4db5-afd2-7b98438cd174 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.153035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 432/847] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:15.153035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646e2cb-6a94-4db5-afd2-7b98438cd174 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.199345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 416/848] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:15.200512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.230409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.265575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646e2cb-6a94-4db5-afd2-7b98438cd174 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.297496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.304437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.305360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.305662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.305937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.306609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:15.306910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.308726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c32cd-64a3-40a3-80a0-09a68e6e97d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.317674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 433/849] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:15.318881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.319163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.319429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.325629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bab8d-b3ab-4ead-becc-82870353ef80 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.339777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:15.341511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:15.362520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.363380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.363654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.363933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.364578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:15.364880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.366296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.383655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 417/850] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:15.383655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.422200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 434/851] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:15.423056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.469428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.489329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17ddf9-8fbb-4ae2-94b9-9c54df092b4b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.524191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.524819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.525139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.525418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.536410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.537915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.541253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fa1e1-c26a-4a58-93fe-7dbb4de78a4e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:15.553388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:15.554377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:15.555281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:15.555281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:15.555281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:15.555281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 418/852] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:15.555940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.571818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.591167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.591802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.592092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.592334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.601336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.603451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:15.616449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:15.617427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:15.618867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 435/853] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:15.619378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.667989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbaa6b-de3d-4530-b4cf-2336a7cf8573 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.704664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.705348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.705617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.705889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.706466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.707832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.709664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:15.718472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:15.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:15.720332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 419/854] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:15.721172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.731046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cd0d8-7b72-47bd-b58f-c2ddafcce1aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.764893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.765547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.765827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.766101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.766765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:15.767078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.768480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.782650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 436/855] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:15.783528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.827690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.828341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec37fe-389d-4f39-b8bb-390151b471af None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.862330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.862927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.863201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.863471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.864159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:15.864462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.865919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.877952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 420/856] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:15.878743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.883499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453bde6-6e7d-472e-beaa-40751f21e205 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:15.921440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:15.922065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:15.922342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:15.922612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:15.923195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:15.924545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:15.933272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:15.934742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:15.935640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 437/857] 10.222.0.85 () {56 vars in 1132 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:15.936434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:15.940504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:15.982780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34c19-c003-4850-b9db-024f7097a284 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.017472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.018083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.018364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.018635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.019264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:16.019566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.020932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.025320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 421/858] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:16.026098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.036750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a8df-91bf-4da8-8db9-6dd50b7c2133 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.043805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:16.072500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.073226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.073433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.073700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.081280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.082647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:16.094077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:16.095240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 438/859] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:16:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:16.096187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.127665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6bfb3-1833-4cba-b130-82b88fe2b9b8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.151511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:16.163660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.164265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.164543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.164814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.165568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=ef33e7dcb56a481aa051c09d45bfc83d) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:16.165877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.167320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.173235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 422/860] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/role_assignments?user.id=ef33e7dcb56a481aa051c09d45bfc83d&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:16.174059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.203791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2606-0808-4dee-a891-d566af245cad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.242118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.242857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.243134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.243403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.244047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:16.244350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.245895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.257356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 439/861] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:16.258256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.295156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:16.386454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d7dab-790e-4798-b299-3822559e6639 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.419927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e53a4f-5728-435a-9203-02d4568361df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.426962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.427577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.427854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.428123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.428854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:16.429184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.430598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.454826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 440/862] 10.222.0.85 () {56 vars in 1149 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:16.455350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.455720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.456150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.456257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.456490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.457158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:16.457459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.459055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.466451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 423/863] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:16.466613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.531745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:16.577257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:16.598606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35681b80-1c37-4ca7-a78a-0379d7563a87 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.599210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47148b3-008a-450b-aa2d-b6db2ae242e4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:16.640253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.640770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.641071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.641343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.641934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:16.642055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:16.642263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.643730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:16.644074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:16.644176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.644418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:16.651032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 441/864] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:16.731800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:16.733313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:16.790818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:16.852132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 424/865] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:16.858685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:16.982477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fec43-bcda-45e5-91f7-16a8b4368b6b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:17.020651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:17.021382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:17.021655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:17.021920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:17.033768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:17.035301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:17.051101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:17.052123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:17.053707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:17.053707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:17.053707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:17.053707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 442/866] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:17.061693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:17.174545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d39eb-5326-469c-b232-f61e54722936 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:17.221108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:17.221713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:17.221988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:17.222248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:17.222885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:17.223179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:17.224526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:17.236810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 425/867] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:17.242189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:17.246150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:17.351353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cfe1e-8c18-41d8-b2ab-ca58dedd2545 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:17.386836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:17.387986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:17.388258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:17.388523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:17.395569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:17.396984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:17.409163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:17.411322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:17.413541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 443/868] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:17.416657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:17.429346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:17.529368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d271e-2a6a-4f86-aec7-1ff7e29ad5e9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:17.564877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:17.565586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:17.565839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:17.566106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:17.566794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:17.567153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:17.568646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5c424-807a-46b6-8b84-1f8b6e7c4c19 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:17.582854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 426/869] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:17.590016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:17.659477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:17.701576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9ac98-60ad-4bb0-8119-e0feefec97b1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:17.736467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:17.737093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:17.737368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:17.737636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:17.738424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=b5445ced373148aeb5741e1e11e897cb) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:17.738732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:17.740133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64907666-ef59-489a-bdb2-5f2a7ad25cff None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:17.746560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 444/870] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:17 2022] GET /identity/v3/role_assignments?user.id=b5445ced373148aeb5741e1e11e897cb&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:18.429710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b0936-faea-4e37-b994-afd0d18f9d69 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:18.430435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b0936-faea-4e37-b994-afd0d18f9d69 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:18.430435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b0936-faea-4e37-b994-afd0d18f9d69 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:18.431835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 427/871] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:18.437390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:18.438342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:18.438833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:18.457964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:18.459546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:18.535147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 445/872] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:18.802595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b0936-faea-4e37-b994-afd0d18f9d69 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:18.891277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:18.931855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b0936-faea-4e37-b994-afd0d18f9d69 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:18.972308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:18.972943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:18.973244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:18.973508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:18.974103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:18.975472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-77ca3d74-57c0-4081-a687-a881221b6c6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:18.985189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:18.987533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:18.988573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 428/873] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:18.992101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:18.994558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdab2e-da47-4b56-8b24-03310496d465 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:18.994850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdab2e-da47-4b56-8b24-03310496d465 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:18.995123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdab2e-da47-4b56-8b24-03310496d465 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:18.996744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 429/874] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:19.002628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae111e-6dd4-43d2-9d86-3bedec10f1ac None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.002904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae111e-6dd4-43d2-9d86-3bedec10f1ac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.003180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae111e-6dd4-43d2-9d86-3bedec10f1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.026533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c3ae111e-6dd4-43d2-9d86-3bedec10f1ac None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:19.027751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c3ae111e-6dd4-43d2-9d86-3bedec10f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:19.062515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:19.108163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2acd3-9743-45d0-b654-81acba319381 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:19.113248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 430/875] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:19.114331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0cd86-7524-481f-a309-466eb3f6ebd9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.114610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0cd86-7524-481f-a309-466eb3f6ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.114876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0cd86-7524-481f-a309-466eb3f6ebd9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.116207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 431/876] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:19.122582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.122850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.123122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.145878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:19.147237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:19.173356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:19.174991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.175886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.176395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.177558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:19.178569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:19.181982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:19.197374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 446/877] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:19.199388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:19.222474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:19.236445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 432/878] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:19.324915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3816ea-10c5-4cfc-94e5-fdfceecbed7e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:19.349413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:19.363559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:19.364502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.365042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.365520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.366351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:19.366855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:19.368415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:19.374506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 447/879] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:19.409790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:19.532176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:19.535947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67caef01-ebab-4de2-bf0c-e85d048d93af None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:19.576307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e5641-8c5e-42fc-b6ba-4eabef1a601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:19.576899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5641-8c5e-42fc-b6ba-4eabef1a601f None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.577205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5641-8c5e-42fc-b6ba-4eabef1a601f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.581211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5641-8c5e-42fc-b6ba-4eabef1a601f None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.581211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e5641-8c5e-42fc-b6ba-4eabef1a601f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:19.581211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e5641-8c5e-42fc-b6ba-4eabef1a601f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:19.599305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 433/880] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:19 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:19.669305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594b882-e9ef-421c-b27b-368989cb7031 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:19.720883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ebcbb-5023-41ad-a8a6-befc62e8c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:19.721726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ebcbb-5023-41ad-a8a6-befc62e8c491 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:19.722067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ebcbb-5023-41ad-a8a6-befc62e8c491 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:19.722342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ebcbb-5023-41ad-a8a6-befc62e8c491 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:19.723015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ebcbb-5023-41ad-a8a6-befc62e8c491 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:19.724479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ebcbb-5023-41ad-a8a6-befc62e8c491 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:19.744619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 448/881] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:19 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:20.066542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d0166-3261-4d6f-9334-229c47e3fcf7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:20.066928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d0166-3261-4d6f-9334-229c47e3fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:20.067125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d0166-3261-4d6f-9334-229c47e3fcf7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:20.068719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 434/882] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:20.074387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:20.074680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:20.074957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:20.095215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:20.096439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:20.134854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:20.181128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 449/883] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:20.441888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d0166-3261-4d6f-9334-229c47e3fcf7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:20.507890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:20.558918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d0166-3261-4d6f-9334-229c47e3fcf7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:20.597665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:20.598298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:20.598570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:20.598839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:20.610960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:20.615897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:20.616815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:20.617839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 435/884] 10.222.0.85 () {56 vars in 1120 bytes} [Wed Nov 2 12:16:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:20.625042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:20.732075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669edc1-289b-46e1-aef6-bef541dc8519 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:20.766847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:20.767831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:20.768303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:20.768790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:20.769731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:20.770235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:20.771788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:20.785110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 450/885] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:16:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:20.787286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:20.896610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5eb8b-521c-4e4a-857b-1f635b952201 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:20.912059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:20.936276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:20.936931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:20.937241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:20.937512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:20.951078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:20.952571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:20.969867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:20.970877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:20.972146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:20.972146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:20.972146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:20.972146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 436/886] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:20.978333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:21.010592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:21.084062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa217-c9c6-4758-b04e-4f89cac2aaa0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:21.120165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:21.121149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.121624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.122178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.123009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:21.123507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:21.125063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:21.142958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 451/887] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:21.144621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:21.257236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb7021-54b3-4aa3-86ac-9da0562fba91 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:21.293545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:21.295143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.295615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.296129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.304231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:21.305980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:21.317721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:21.318703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:21.319634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:21.326585 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 437/888] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:16:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:21.328134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:21.424167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:21.436751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bc1b2-0d32-47c9-9383-a06da12b25a0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:21.474742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:21.475354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.475626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.475893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.476620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:21.476921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:21.478462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9b6bd-ec6f-4fde-8aa2-627f7953d082 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:21.502702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 452/889] 10.222.0.85 () {56 vars in 1149 bytes} [Wed Nov 2 12:16:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:21.504177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:21.535242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:21.641241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f166-aebe-4b9d-87a3-d12488603152 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:21.678391 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cd6ab-d4c8-453e-ae06-5a63132520b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:21.679377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cd6ab-d4c8-453e-ae06-5a63132520b3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.679875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cd6ab-d4c8-453e-ae06-5a63132520b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.680341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cd6ab-d4c8-453e-ae06-5a63132520b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.734816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595364-e7fa-4fbd-81a0-8a4a2138e184 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.735374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595364-e7fa-4fbd-81a0-8a4a2138e184 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.735374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595364-e7fa-4fbd-81a0-8a4a2138e184 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.736980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 453/890] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:21.742439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.742730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.742979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.758670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cd6ab-d4c8-453e-ae06-5a63132520b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:21.762990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cd6ab-d4c8-453e-ae06-5a63132520b3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:21.766073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:21.767339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:21.806454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:21.865652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 454/891] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:21.910551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 438/892] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:21 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:21.916010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:21.981267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc325-5b10-43d2-ac26-5d0f518342a1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.981652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc325-5b10-43d2-ac26-5d0f518342a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.981859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc325-5b10-43d2-ac26-5d0f518342a1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.983209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 439/893] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:21.984264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ffc2e-1dfe-4e42-9f8f-1531684ba1de None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.984536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ffc2e-1dfe-4e42-9f8f-1531684ba1de None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.984802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ffc2e-1dfe-4e42-9f8f-1531684ba1de None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:21.986267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 440/894] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:21.988680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426dd7-1501-4aa7-8eee-892707e80667 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:21.988959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426dd7-1501-4aa7-8eee-892707e80667 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:21.989277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426dd7-1501-4aa7-8eee-892707e80667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.010762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.011140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-34426dd7-1501-4aa7-8eee-892707e80667 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:22.012686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-34426dd7-1501-4aa7-8eee-892707e80667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:22.035644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947b023-c674-4872-ad25-8577cdf11035 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.081577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.082617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.083067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.083521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.095000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 441/895] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:22.097591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.098356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.099082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.099640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.101110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:22.118342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:22.128009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:22.130045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:22.130045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:22.130045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:22.130045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 455/896] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:22.130482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:22.130482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:22.137044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.202206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.225060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 442/897] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:22.225060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.262519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a5cc9-a954-47dc-b188-5b19a1e41760 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.305344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.314439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.316473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.317490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.317860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.318604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:22.318974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.320446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.336138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 456/898] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:22.341557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.375576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ff8c4-3236-405c-99fa-a8fc5e95a186 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.412274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.412879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.413207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.413457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.414117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.415481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.433099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 443/899] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:22 2022] POST /identity/v3/services => generated 241 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:22.433099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.472544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941267a-9c14-4955-a47a-2cc35c178539 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.514283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.522184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.522835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.523114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.523383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.531023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.532552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:22.545432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:22.546512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 457/900] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:22.547430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.592453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95f9e9-e56b-4ba6-9949-689acfecb229 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.608375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.633292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.634499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.634499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.634499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.635024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.636380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:22.645842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:22.650267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:22.651106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 444/901] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:22.651889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.675345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49877935-6ada-4100-a7b1-2b2cb368a0ee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.710881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.711577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.711858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.712126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.712718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.714264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:22.724091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:22.725096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:22.726019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 458/902] 10.222.0.85 () {56 vars in 1132 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:22.726840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.770503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8374bd-6331-4c41-a685-5191ed5bf17b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.782653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.808389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.809086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.809310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.809630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.810327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:22.810678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.812126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.829042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 445/903] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:22.829042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.842981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e691f23-f398-43f7-84c3-6e9042a1dfc6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.855191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:22.884614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.885253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.885525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.885791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.886415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:22.886714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.888097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.893402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 459/904] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:22.894206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:22.939822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6e419-a2f5-491f-8362-29624f239fe2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:22.977575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:22.978548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:22.979057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:22.979531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:22.980359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:22.980853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:22.983531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:22.990200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 446/905] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:22.991355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:23.007367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a3b01-dc5c-4c1a-8db6-5eff0e4fa33b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.036317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:23.045768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.046709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.046983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.047254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.047980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] RBAC: Adding query filter params (role.id=2e3744c807604e12b85f8c897ed8316f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=b5445ced373148aeb5741e1e11e897cb) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:23.048289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.049946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.056312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 460/906] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/role_assignments?user.id=b5445ced373148aeb5741e1e11e897cb&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=2e3744c807604e12b85f8c897ed8316f => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:23.057221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:23.106207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:23.148141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b42f11-d502-4eee-acd5-2d7e48feed3f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.180154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3e70-e5db-4875-978b-1518e9aa6e23 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.201024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.202496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.203001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.203495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.204357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:23.205367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.206780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.222794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.223474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 447/907] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:23.224428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.224941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.225453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.226299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:23.226823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.228622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.229259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:23.236540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 461/908] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:23.242775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:23.342366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:23.346645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54801a4-2c3a-4198-b6da-114130719c3b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.370568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac72bc-f654-44d4-b9e9-b4a4c265fcd6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.372185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:23.389796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.390517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.390903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.391240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.392110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] RBAC: Adding query filter params (interface=public, service_id=2b7d34ac93b2407aafca63b1841487fb, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:23.392578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.394053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.402831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 448/909] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:23 2022] GET /identity/v3/endpoints?service_id=2b7d34ac93b2407aafca63b1841487fb&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:23.409162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:23.411106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.411723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.412002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.412387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.413245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] RBAC: Adding query filter params (interface=public, service_id=121682b4c7174c62a1e218872c41da13, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:23.413562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.414933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.420946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 462/910] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:23 2022] GET /identity/v3/endpoints?service_id=121682b4c7174c62a1e218872c41da13&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:23.427320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:23.525508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46449e38-a4c7-4776-a6b1-e23cdd62fa0e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.559586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3359a-7236-4c7e-ae96-1be9c7e8b1cd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:23.567091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744c9e3-1017-46e0-9935-52f9cb9a5da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.568142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744c9e3-1017-46e0-9935-52f9cb9a5da2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.568433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744c9e3-1017-46e0-9935-52f9cb9a5da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.568706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744c9e3-1017-46e0-9935-52f9cb9a5da2 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.569409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744c9e3-1017-46e0-9935-52f9cb9a5da2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.570787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744c9e3-1017-46e0-9935-52f9cb9a5da2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.583410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 449/911] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:23 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:16:23.608311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3a886-d3c3-4fbb-a8af-a35cec3f0782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:23.609024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3a886-d3c3-4fbb-a8af-a35cec3f0782 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:23.609319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3a886-d3c3-4fbb-a8af-a35cec3f0782 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:23.609658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3a886-d3c3-4fbb-a8af-a35cec3f0782 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:23.610350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3a886-d3c3-4fbb-a8af-a35cec3f0782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:23.611772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3a886-d3c3-4fbb-a8af-a35cec3f0782 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:23.625200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 463/912] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:23 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:16:24.648369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b8fb5-d66d-4e58-a1f1-460b57c7ee22 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:24.648918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b8fb5-d66d-4e58-a1f1-460b57c7ee22 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:24.648918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b8fb5-d66d-4e58-a1f1-460b57c7ee22 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:24.650448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 450/913] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:24.656475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:24.656872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:24.657079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:24.677578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:24.679074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:24.704801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:24.751727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 464/914] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:25.022958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b8fb5-d66d-4e58-a1f1-460b57c7ee22 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.055795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:25.138657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b8fb5-d66d-4e58-a1f1-460b57c7ee22 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:25.177041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:25.178082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.178554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.179038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.179900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:25.181641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ec88b7dc-7a0c-4664-bcaf-f39e7e5812ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:25.192277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:25.193457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:25.194401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:25.197159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 451/915] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:25.205202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.308902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153cb7-36ae-4ae3-ae49-4f953b228876 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.309311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153cb7-36ae-4ae3-ae49-4f953b228876 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.309688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153cb7-36ae-4ae3-ae49-4f953b228876 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.311435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 452/916] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:25.312251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e358a53-c7e3-4116-a27a-614992c1b994 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:25.317029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88855412-c562-4420-977b-cde98a648bc3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.317305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88855412-c562-4420-977b-cde98a648bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.317578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88855412-c562-4420-977b-cde98a648bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.340460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-88855412-c562-4420-977b-cde98a648bc3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:25.342609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-88855412-c562-4420-977b-cde98a648bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:25.357713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:25.358873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.359753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.360516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.361398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:25.362055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:25.363941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:25.371172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 465/917] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:25.376392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.408160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:25.421819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 453/918] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:25.492508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1d97f-bf51-4796-aaa7-0759c86846a3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:25.529855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:25.530465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.530739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.531006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.531640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:25.531938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:25.533317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:25.538977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 466/919] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:25.544460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88855412-c562-4420-977b-cde98a648bc3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.567015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:25.655695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88855412-c562-4420-977b-cde98a648bc3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:25.697576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:25.698200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.698476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.698767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.699479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] RBAC: Adding query filter params (interface=public, service_id=0315166b776a42f4ab1334196c6834f7, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:25.699790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:25.701750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:25.708200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 454/920] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/endpoints?service_id=0315166b776a42f4ab1334196c6834f7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:25.718199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.732200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.829966 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:25.841279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:25.848269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7071f-19ab-400b-a912-7ce7c5825f84 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:25.857048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c098d08-f619-43f8-872b-b297067a4fe9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:25.886171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:25.886795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.887067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.887564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.895834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae0168-98ce-498f-a555-820cc8a4663c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:25.896441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae0168-98ce-498f-a555-820cc8a4663c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.896719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae0168-98ce-498f-a555-820cc8a4663c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.897107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae0168-98ce-498f-a555-820cc8a4663c None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.897717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae0168-98ce-498f-a555-820cc8a4663c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:25.899004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:25.899118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae0168-98ce-498f-a555-820cc8a4663c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:25.903802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:25.904901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:25.905892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:25.906562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 467/921] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:16:25.906562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7a53a-c500-4672-95b0-b0f383440187 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.906562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7a53a-c500-4672-95b0-b0f383440187 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.906859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7a53a-c500-4672-95b0-b0f383440187 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.913976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 468/922] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:25.913976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb91f7-ca44-4834-85ed-62fbf38153ef None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.913976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb91f7-ca44-4834-85ed-62fbf38153ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.913976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb91f7-ca44-4834-85ed-62fbf38153ef None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.913976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 469/923] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:25.914523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 455/924] 10.222.0.85 () {56 vars in 1120 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:25.914604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6160-32cb-4fc0-97a8-20d2b96653e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:25.915154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6160-32cb-4fc0-97a8-20d2b96653e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:25.915732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6160-32cb-4fc0-97a8-20d2b96653e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:25.919396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:25.938443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-675c6160-32cb-4fc0-97a8-20d2b96653e7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:25.939666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-675c6160-32cb-4fc0-97a8-20d2b96653e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:26.025087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 470/925] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:26.025087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.025087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.025087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.045305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:26.046883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:26.047967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4601250-48e3-424c-b20a-1ed308041d22 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.093456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.094075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.094348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.094615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.095278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:26.095575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.096941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.105694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 456/926] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:16:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:26.111695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.123057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.137061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 471/927] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:26.229466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a6b1d-5ce8-4aac-8a26-218bc2927bdf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.266471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.267157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.267433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.267701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.279138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.280640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:26.295965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:26.296965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:26.298540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:26.298540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:26.298540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.298540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 457/928] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:26.306093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.307699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.322324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.375828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.421680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f81e0a-2e8a-4b85-af5b-aec78b99a10e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.447183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd0bb9-8988-4fd9-88c9-1a2f1219a72d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.456692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.457339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.457702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.457972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.458553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.459918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:26.470227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:26.471193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 12:16:26.472039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.476474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 458/929] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:26.477218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.487481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.488173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.488512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.488847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.489582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:26.489948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.491355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.509033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 472/930] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:26.509879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.585656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603119a2-6d31-47a7-b71d-c80fdb67d707 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.589251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.614891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.620885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc52b6e-111c-48b9-b390-aa0410b525ee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.627721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.628369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.629042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.629042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.629719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.631337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:26.640646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:26.641847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 12:16:26.642942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.648430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 459/931] 10.222.0.85 () {56 vars in 1132 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:26.649438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.660467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.661182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.661524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.661803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.662434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:26.662736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.664095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.669349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 473/932] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:26.670215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.760883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c08f-0a9c-459c-bbed-f315b14a64ba None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.784366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720143c-a9c7-4ef6-b206-71590e4154c5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.802898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.803810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.804277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.804743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.814063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.815684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.820486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.821120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.821397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.821662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.822283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:26.822580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.823910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.826094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:26.827691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:26.829357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 460/933] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:26.830343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.835331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 474/934] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:26.836095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.884496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:26.942967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25214ed-34b1-4bc7-8503-d805d798dcb2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.953700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8dd30-736f-4123-b20f-c90dd8c1b882 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:26.979433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.980147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.980492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.980769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.981610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:26.982400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.983762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:26.993312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 461/935] 10.222.0.85 () {56 vars in 1114 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:26.993312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:26.994675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:26.995291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:26.995568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:26.995838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:26.996703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:26.997151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:26.999166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.023442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 475/936] 10.222.0.85 () {56 vars in 1149 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:27.024338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:27.093218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:27.098844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798cb2b-024c-44b3-aacc-aeb0ec3886e4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.110252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:27.139849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be692e3f-7126-40e1-9d01-e57ed56481bb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.140153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.140694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.140965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.141266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.141892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:27.142189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.143547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.148462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 462/937] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:27.149294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:27.186623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.187577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.188043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.188396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.189172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.190546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.216172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 476/938] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:27.217659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:27.307337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad202fb-5458-4b3e-8f1c-6a33e8981e92 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.342416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5cf85-daeb-4295-bd77-4b719406688f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.355588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.356822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.358314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.358821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.362451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:27.375582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.378975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.383824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.384444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.384726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.385024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.385768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.387222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:27.400661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:27.402264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:27.403198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:27.403198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:27.403198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:27.406471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 463/939] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:27.411961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:27.418520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 477/940] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:27.448714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:27.540699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edea1-b629-4643-ab3e-d16bc19d21bf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.589940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.590734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.591107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.591543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.592336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:27.592729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.594283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.608872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 464/941] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:27.614152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:27.728758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db10054-6cbd-4cfe-a0a5-b276584bffad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.766507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.767645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.767986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.768491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.776476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.778223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:27.792257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:27.793655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:27.794629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 478/942] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:27.800630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:27.899510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:27.922749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289d058-9333-4199-975b-cd3d4c9d7f57 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:27.965356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:27.965969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:27.966239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:27.966542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:27.967237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:27.967537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:27.969030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6de426-1f57-4b1e-abf6-c03962376891 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:27.984042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 465/943] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:27.989922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:28.036531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:28.106933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3e05-0f21-4df7-bd94-c27df14b31a0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:28.147623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdf8e8-3a01-496f-8edd-f4f4e4c3d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:28.148295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdf8e8-3a01-496f-8edd-f4f4e4c3d8f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:28.148569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdf8e8-3a01-496f-8edd-f4f4e4c3d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:28.148838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdf8e8-3a01-496f-8edd-f4f4e4c3d8f7 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/b5445ced373148aeb5741e1e11e897cb/roles/2e3744c807604e12b85f8c897ed8316f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:28.151088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdf8e8-3a01-496f-8edd-f4f4e4c3d8f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=b5445ced373148aeb5741e1e11e897cb, role_id=2e3744c807604e12b85f8c897ed8316f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:28.152515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdf8e8-3a01-496f-8edd-f4f4e4c3d8f7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:28.169526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 479/944] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:16:27 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/b5445ced373148aeb5741e1e11e897cb/roles/2e3744c807604e12b85f8c897ed8316f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:16:28.233496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dec76-b0c5-46d5-a992-e3c0d2107978 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:28.233769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dec76-b0c5-46d5-a992-e3c0d2107978 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:28.234032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dec76-b0c5-46d5-a992-e3c0d2107978 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:28.235816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 466/945] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:28.243085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:28.243455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:28.243790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:28.266360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:28.267761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:28.347507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 480/946] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:28.621867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dec76-b0c5-46d5-a992-e3c0d2107978 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:28.736609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dec76-b0c5-46d5-a992-e3c0d2107978 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:28.774196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:28.774831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:28.775292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:28.775681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:28.776272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:28.777735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:28.787655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:28.788666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:28.789553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 467/947] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:28.795720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:28.813430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:28.902287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a00154-5b38-4122-aab8-4c7cbd5728cd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:28.943379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:28.944375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:28.944892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:28.945407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:28.946254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:28.946762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:28.948310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:28.953856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 481/948] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:28.959339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:28.969737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:29.069571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9513bfc-2d3d-49aa-ab67-7b716cfcc2c1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:29.109712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:29.110321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:29.110592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:29.110854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:29.111487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:29.111782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:29.113289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531c188-36b1-444b-b9b5-d8c5e0680efb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:29.118979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 468/949] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:29.124785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:29.215484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:29.236763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaa910-a301-48bf-8019-7fd36395b8ef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:29.273103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14771-06e5-4dd4-a5db-78ebd8f7bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:29.273844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14771-06e5-4dd4-a5db-78ebd8f7bf98 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:29.274124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14771-06e5-4dd4-a5db-78ebd8f7bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:29.274397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14771-06e5-4dd4-a5db-78ebd8f7bf98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:29.275087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14771-06e5-4dd4-a5db-78ebd8f7bf98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:29.276536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14771-06e5-4dd4-a5db-78ebd8f7bf98 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:29.299625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 482/950] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:29.606724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d654b1-d606-4807-8b60-8802446da355 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:29.607109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d654b1-d606-4807-8b60-8802446da355 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:29.607832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d654b1-d606-4807-8b60-8802446da355 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:29.609460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 469/951] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:29.615038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e087f83-4d61-4690-a964-8d36cafd7d09 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:29.615339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e087f83-4d61-4690-a964-8d36cafd7d09 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:29.615616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e087f83-4d61-4690-a964-8d36cafd7d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:29.634551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5e087f83-4d61-4690-a964-8d36cafd7d09 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:29.635762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5e087f83-4d61-4690-a964-8d36cafd7d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:29.719154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 483/952] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:29.835409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091beb51-3e3e-43da-abdd-4170cc4729fe None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:29.835777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091beb51-3e3e-43da-abdd-4170cc4729fe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:29.836152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091beb51-3e3e-43da-abdd-4170cc4729fe None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:29.837887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 470/953] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:29.843813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:29.844266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:29.844342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:29.863196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:29.864460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:29.925941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:29.941271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 484/954] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:29.993415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091beb51-3e3e-43da-abdd-4170cc4729fe None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.080449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:30.114949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091beb51-3e3e-43da-abdd-4170cc4729fe None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:30.152053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:30.153063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.153549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.154076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.155417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:30.157069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:30.159992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING keystone.server.flask.common [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 12:16:30.178853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.sql.core [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 12:16:30.178853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 12:16:30.178853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [parameters: {'id': '6536880855504a9295d6aeee1bede591', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 12:16:30.178853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6536880855504a9295d6aeee1bede591.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6536880855504a9295d6aeee1bede591. Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 12:16:30.197613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 12:16:30.198511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application yield resource Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise self.value Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application session.commit() Nov 02 12:16:30.199476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.flush() Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application compat.raise_( Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 12:16:30.200394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 12:16:30.203114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 12:16:30.205128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [parameters: {'id': '6536880855504a9295d6aeee1bede591', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.207258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:30.209217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6536880855504a9295d6aeee1bede591. Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.211399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 471/955] 10.222.0.85 () {60 vars in 1140 bytes} [Wed Nov 2 12:16:29 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:30.219959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.236787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.356143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e0e0-04f2-4fb1-a638-7d67341276d7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:30.385384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c425-22e8-4472-9768-07a8fe72f6d5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:30.403607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:30.404934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.405724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.406172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.414840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:30.417075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:30.417517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:30.427710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:30.428341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.428618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.428889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.429518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:30.431113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-906dcc1a-a976-4e64-bd14-d36434557043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:30.431313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:30.432189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:30.433204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 485/956] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:30.434236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac453c1a-65ca-4232-a8be-e02a4978d103 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.434236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac453c1a-65ca-4232-a8be-e02a4978d103 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.434236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac453c1a-65ca-4232-a8be-e02a4978d103 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.436306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 486/957] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:30.441554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac453c1a-65ca-4232-a8be-e02a4978d103 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ad96eebc-1b44-4719-8850-239a4c73a59f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:30.448872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:30.450077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:30.451526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 472/958] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:30.452907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.457403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.457403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.475964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:30.477503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:30.503105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:30.557075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 473/959] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:30.557075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.567970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac453c1a-65ca-4232-a8be-e02a4978d103 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:30.611153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:30.612058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.612502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.612923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.613760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:30.614365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:30.616850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:30.637850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 487/960] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:30.711574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f0d3b-32bc-4ace-9da1-b0a7e8b9ebf4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:30.737134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:30.757548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:30.758219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.758502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.758776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.759432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] RBAC: Adding query filter params (name=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:30.759739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:30.761179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:30.769531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 474/961] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:30.775743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.859331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:30.905992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a51dd-36c2-4ebc-ae0d-a3fe3d92b7e0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:30.925092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:30.954409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:30.955046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:30.955326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:30.955599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:30.956258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:30.956567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:30.958088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:30.969157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 488/962] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:31.007634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7a4f9-a75d-4ae5-9a42-64e68baa6dde None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:31.052228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:31.052891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.053205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.053479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.067236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:31.072869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:31.074181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:31.075087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 475/963] 10.222.0.85 () {56 vars in 1120 bytes} [Wed Nov 2 12:16:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:31.081237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:31.202505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f863b3-5608-48a8-840f-210cbcb6e787 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:31.237613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:31.238224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.238567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.238843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.239517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:31.239819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:31.241220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:31.255523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 489/964] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:16:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:31.256895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:31.312178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:31.378380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ab1d-e68f-45f6-9818-c599e82d5e79 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:31.415889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:31.416553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.416833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.417146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.430594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:31.432093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9a334912-2a92-4e54-a8a2-ab1faf854c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:31.449288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:31.450380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:31.451639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:31.451639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:31.451639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:31.451639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 476/965] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:31.457762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:31.503753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:31.534658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619117-6beb-4329-b14c-89832454d4d9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.534986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619117-6beb-4329-b14c-89832454d4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.535201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619117-6beb-4329-b14c-89832454d4d9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.536900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 477/966] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:31.542017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.542294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.542565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.563489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:31.564689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:31.572298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcc3cc-8334-4920-843a-6c31d092d3de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:31.613019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:31.613746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.614093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.614436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.615155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:31.615528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:31.617132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:31.632766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 490/967] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:31.638490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:31.658485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 478/968] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:31.756471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:31.761969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84aa485-c58a-4b34-94c9-c4c345214795 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:31.800346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:31.801426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.801942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.802418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.810938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:31.812663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:31.825198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:31.826847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 12:16:31.827844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:31.834759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 491/969] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:16:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:31.838093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:31.867282 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:31.931749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:31.953620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60c980-da96-4c0f-8779-a83266a9d1f2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:31.990807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:31.991440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:31.991715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:31.991986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:31.992768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:31.993093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:31.994599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.019547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 479/970] 10.222.0.85 () {56 vars in 1149 bytes} [Wed Nov 2 12:16:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:32.021712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.063099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7604cf-541e-4c8d-ab10-a2b98001561b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.107978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.109250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.110459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.110970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.112072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.114943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:32.128025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:32.129127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:32.129983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:32.132890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.134040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 492/971] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:32.140506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.155656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd2b11-38b5-4c46-be6e-3125b55bea52 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.177268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.195213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.196112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.196531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.196940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.269855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.271211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.282326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3c576-3046-4a5b-9c7e-bd0796d1a4ce None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.323622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.324227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.324500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.324765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.325431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:32.325831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.327204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.336671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 493/972] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:32.337636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.373587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.392172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 480/973] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 12:16:32.397757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.449247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.469372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70d932-cdf5-4a77-ba00-d72df7f7a645 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.510757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.511518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.511884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.512427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.512961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:32.513506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.514850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.519649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04799-6755-4111-8ed6-abfca17b58f7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.525325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 494/974] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:32.526949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.564318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.565242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.565669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.566088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.579516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.581844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:32.603188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:32.604407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:32.605565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:32.605565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:32.605565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:32.607928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 481/975] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:32.613395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.645514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fac97-dd96-468a-9adf-e9fe0c610a89 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.661149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.687464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.688258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.688393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.688619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.689399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] RBAC: Adding query filter params (interface=admin, service_id=0315166b776a42f4ab1334196c6834f7, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:32.689829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.691230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.701089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 495/976] 10.222.0.85 () {56 vars in 1249 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/endpoints?service_id=0315166b776a42f4ab1334196c6834f7&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:32.703527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.726507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.759618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6869d5c-57d6-4f56-900a-6963b781835c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.805067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.805604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.805883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.806155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.806805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:32.807163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.809271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.825872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 482/977] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:32.831683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:32.838898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84318d14-7e55-4ad5-872a-5958680bedee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:32.882072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:32.883500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:32.884317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:32.884808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:32.885659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:32.887720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:32.896714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 496/978] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:16:32.952115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:32.959792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78b330-6f64-4812-85dc-52ef2546eff4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:33.002640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:33.003283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.003553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.003829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.012032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:33.014173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ba1d79d9-1845-40a0-a263-238cec847142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:33.026524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:33.027912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:33.029031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 483/979] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:33.036701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:33.084509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7068-266c-4cbc-9285-d980b3e7d9a4 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.084819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7068-266c-4cbc-9285-d980b3e7d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.085047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7068-266c-4cbc-9285-d980b3e7d9a4 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.086932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 484/980] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:33.093563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.094155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.094648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.116840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:33.118407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:33.136297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:33.164655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca753ca-a869-49dd-99dd-392a04831901 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:33.207351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:33.207979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.208254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.208519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.209301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:33.209605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:33.211111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42981-e2da-493b-a755-34ea59f1c28b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:33.217026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 485/981] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:33.233226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 497/982] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:33.235091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:33.332058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03653-7a2b-457b-916b-eecd9179cc9c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.332389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03653-7a2b-457b-916b-eecd9179cc9c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.332608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03653-7a2b-457b-916b-eecd9179cc9c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.334486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 498/983] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:33.340080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:33.341043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.342218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.342728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.352204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525f800-0d69-40bc-aaf6-1bacbb0bb83a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:33.369119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:33.370813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:33.397315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:33.398215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.398489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.398758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.399485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] RBAC: Adding query filter params (role.id=2e3744c807604e12b85f8c897ed8316f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=b5445ced373148aeb5741e1e11e897cb) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:33.400057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:33.402202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:33.417113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 486/984] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:33 2022] GET /identity/v3/role_assignments?user.id=b5445ced373148aeb5741e1e11e897cb&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=2e3744c807604e12b85f8c897ed8316f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:33.456820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 499/985] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:33.493367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:33.615359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071fb94-8040-47cc-af22-50fa22cfa682 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:33.651185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:33.651791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.652060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.652324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.652974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:33.654457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:33.670664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.sql.core [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 12:16:33.670664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 12:16:33.670664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [parameters: {'user_id': '9959f48114644a91b6bf49a7bfb7c73f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 12:16:33.670664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Nov 02 12:16:33.678067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 12:16:33.679771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application yield resource Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise self.value Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application session.commit() Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 12:16:33.682931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.flush() Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application compat.raise_( Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 12:16:33.686649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 12:16:33.689209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 12:16:33.691074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [parameters: {'user_id': '9959f48114644a91b6bf49a7bfb7c73f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:33.701148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 12:16:33.702040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:33.702952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 487/986] 10.222.0.85 () {60 vars in 1131 bytes} [Wed Nov 2 12:16:33 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:33.703423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:33.755946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:33.772073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:33.772387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:33.827951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9062-2b37-4796-9c41-69238ee57995 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:33.876476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:33.877047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.877332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.877640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.889457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:33.890939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:33.891250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4def121-e813-44c2-8f3a-b930f17aae7a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:33.907433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:33.908448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:33.909833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 500/987] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:33.914308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:33.947349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e595d1f-25e4-477b-a023-144563ad8412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:33.948324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e595d1f-25e4-477b-a023-144563ad8412 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:33.949018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e595d1f-25e4-477b-a023-144563ad8412 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:33.949536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e595d1f-25e4-477b-a023-144563ad8412 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:33.950364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e595d1f-25e4-477b-a023-144563ad8412 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:33.952825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e595d1f-25e4-477b-a023-144563ad8412 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:33.985061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 488/988] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:33 2022] POST /identity/v3/services => generated 242 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:34.058201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6573d-f3df-448c-a9a2-7781f6e8f2d4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:34.096791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:34.120715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:34.121396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:34.121687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:34.121937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:34.122648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:34.123817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:34.125372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd2957-6773-47d2-be41-cdcef8a52440 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:34.165078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 501/989] 10.222.0.85 () {56 vars in 1101 bytes} [Wed Nov 2 12:16:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:35.302356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e44f32-74e9-4414-9ab6-87a58609732a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:35.302960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e44f32-74e9-4414-9ab6-87a58609732a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:35.302960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e44f32-74e9-4414-9ab6-87a58609732a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:35.304382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 489/990] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:35.309636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade63fb-a9ae-42f0-8d76-d0d3f000c110 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:35.310094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade63fb-a9ae-42f0-8d76-d0d3f000c110 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:35.310184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade63fb-a9ae-42f0-8d76-d0d3f000c110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:35.328939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bade63fb-a9ae-42f0-8d76-d0d3f000c110 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:35.330182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bade63fb-a9ae-42f0-8d76-d0d3f000c110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:35.400319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 502/991] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:35.668700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e44f32-74e9-4414-9ab6-87a58609732a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:35.726167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9618-8047-4249-851b-85c2821658cf None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:35.726515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9618-8047-4249-851b-85c2821658cf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:35.726736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9618-8047-4249-851b-85c2821658cf None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:35.728092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 503/992] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:35.733859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:35.734139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:35.734592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:35.750453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:35.753230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:35.754398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:35.785554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e44f32-74e9-4414-9ab6-87a58609732a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:35.823798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:35.829218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:35.829369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 504/993] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:35.829465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:35.829465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:35.829465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:35.829465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:35.840506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:35.841567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 12:16:35.842492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:35.845325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 490/994] 10.222.0.85 () {56 vars in 1126 bytes} [Wed Nov 2 12:16:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:35.847566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:35.893751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:35.953893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cc0b8-b3f1-4dc1-9575-6b95c3a3a0be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:35.988437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:35.989056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:35.989333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:35.989596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:35.990210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:35.990505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:35.991836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.001944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 505/995] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:36.002905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.094510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.111900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dfae3-a44c-475f-956a-f332b19bfa0c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:36.151431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:36.152035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.152305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.152571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.153223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:36.153519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:36.154914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.157083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:36.164754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 491/996] 10.222.0.85 () {56 vars in 1122 bytes} [Wed Nov 2 12:16:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:36.165747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.207644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346fd40-826f-49ab-9d1d-1748083fa917 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:36.215916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:36.244216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:36.244832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.245135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.245404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.245976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:36.247595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f2a76256-fd97-4638-a50a-03681d4ba9ad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:36.256221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:36.257344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:36.258464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 506/997] 10.222.0.85 () {56 vars in 1105 bytes} [Wed Nov 2 12:16:36 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:36.259370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205289-760f-4971-9cc1-2e36357426af None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.259370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205289-760f-4971-9cc1-2e36357426af None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.259370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205289-760f-4971-9cc1-2e36357426af None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.260541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 507/998] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:36.263423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205289-760f-4971-9cc1-2e36357426af None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.274216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43808085-87c4-433e-9675-0782b8713a4f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:36.309150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55095b91-04be-4979-b408-5436cc66adea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:36.309808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55095b91-04be-4979-b408-5436cc66adea None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.310003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55095b91-04be-4979-b408-5436cc66adea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.310267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55095b91-04be-4979-b408-5436cc66adea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.310952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55095b91-04be-4979-b408-5436cc66adea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:36.312302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55095b91-04be-4979-b408-5436cc66adea None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.330420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 492/999] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:16:36.331624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.331913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.332180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.352639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:36.354272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:36.384458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205289-760f-4971-9cc1-2e36357426af None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:36.424189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:36.424748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:36.425402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.425682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.426062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.426951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] RBAC: Adding query filter params (name=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:36.426951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:36.428528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.434785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 508/1000] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:36.438835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 493/1001] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:36.440560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.478785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:36.558183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca85ced-1010-4809-b2cd-6a7312953199 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:36.595993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:36.596631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.596913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.597215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.597849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] RBAC: Adding query filter params (type=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:36.598154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:36.599506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.604446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 509/1002] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:36.708631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.818801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228e794-8960-4ac8-85cb-2962b21937ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:36.855157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:36.855792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:36.856069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:36.856339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:36.856928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:36.858356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:36.868414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:36.869614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:36.871062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 494/1003] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:36.875805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:36.977714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:36.986022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dd736-1523-430f-a475-b0609a4b56bf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:37.023339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:37.024106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:37.024862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:37.025172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:37.025807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:37.026109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:37.027470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:37.037348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 510/1004] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:37.039065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:37.101968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:37.149908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457684-d250-41f6-85cc-8f6592dec04c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:37.183765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:37.184424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:37.184706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:37.184975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:37.186156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:37.186506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:37.187987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:37.194960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 495/1005] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:37.199385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:37.303252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f7c92-d3c6-4592-ab87-986518d5cc51 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:37.338780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:37.339391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:37.339668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:37.339938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:37.340638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] RBAC: Adding query filter params (interface=public, service_id=7473a21d4e30497a8de7758a33ff4bab, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:37.340941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:37.342358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05ee88-ce41-4860-b047-b3a5bf500590 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:37.347635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 511/1006] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:37 2022] GET /identity/v3/endpoints?service_id=7473a21d4e30497a8de7758a33ff4bab&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:37.353400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:37.459349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef6a29-499b-4b58-97ed-7988c2ebd05c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:37.489774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:37.520564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2d4f6-de38-489d-97ff-69a6fbb7c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:37.521643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d4f6-de38-489d-97ff-69a6fbb7c804 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:37.522138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d4f6-de38-489d-97ff-69a6fbb7c804 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:37.522630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d4f6-de38-489d-97ff-69a6fbb7c804 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:37.523595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2d4f6-de38-489d-97ff-69a6fbb7c804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:37.526227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2d4f6-de38-489d-97ff-69a6fbb7c804 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:37.545509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 496/1007] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:37 2022] GET /identity/v3/services => generated 1727 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:16:38.586951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73c55e-8e4a-470a-9cc2-cda025c9075f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:38.587546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73c55e-8e4a-470a-9cc2-cda025c9075f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:38.587546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73c55e-8e4a-470a-9cc2-cda025c9075f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:38.589284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 512/1008] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:38.594611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f9a2-03e8-4540-a8ba-dd90e722d384 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:38.595067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f9a2-03e8-4540-a8ba-dd90e722d384 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:38.595242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f9a2-03e8-4540-a8ba-dd90e722d384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:38.613333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1269f9a2-03e8-4540-a8ba-dd90e722d384 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:38.614555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1269f9a2-03e8-4540-a8ba-dd90e722d384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:38.688844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 497/1009] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:38.786744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01371b-f583-4580-b597-788d83ab026c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:38.786919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01371b-f583-4580-b597-788d83ab026c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:38.787199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01371b-f583-4580-b597-788d83ab026c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:38.788671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 513/1010] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:38.793854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:38.794136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:38.794406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:38.813612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:38.814788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:38.882565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 498/1011] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:38.957542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01371b-f583-4580-b597-788d83ab026c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:38.981851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:39.061483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01371b-f583-4580-b597-788d83ab026c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:39.094910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:39.095565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:39.095846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:39.096118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:39.097377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:39.098848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:39.100687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 514/1012] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:39.106455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:39.149827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:39.151871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:39.210632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed323-c74c-4426-9e56-674b3cc7358a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:39.247636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05402fde-16bb-44f2-bfd0-44071de292ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:39.248043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:39.248523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05402fde-16bb-44f2-bfd0-44071de292ce None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:39.248806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05402fde-16bb-44f2-bfd0-44071de292ce None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:39.249106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05402fde-16bb-44f2-bfd0-44071de292ce None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:39.249762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05402fde-16bb-44f2-bfd0-44071de292ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:39.251735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05402fde-16bb-44f2-bfd0-44071de292ce None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:39.259007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b249bc-7ca4-4a38-a540-2aa583ea7829 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:39.281470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 499/1013] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:39 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:39.303896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba45657-e7b7-4dc6-b6c9-099425aa1e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:39.304914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba45657-e7b7-4dc6-b6c9-099425aa1e1e None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:39.305370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba45657-e7b7-4dc6-b6c9-099425aa1e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:39.305797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba45657-e7b7-4dc6-b6c9-099425aa1e1e None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:39.306650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba45657-e7b7-4dc6-b6c9-099425aa1e1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:39.308255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba45657-e7b7-4dc6-b6c9-099425aa1e1e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:39.325268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 515/1014] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:16:39 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:16:39.780533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf708f2-d480-48aa-924b-11fb0c12472d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:39.781242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf708f2-d480-48aa-924b-11fb0c12472d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:39.781242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf708f2-d480-48aa-924b-11fb0c12472d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:39.782609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 500/1015] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:39.787927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:39.788359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:39.788554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:39.807213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:39.808408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:39.884916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 516/1016] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:39 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:40.155084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf708f2-d480-48aa-924b-11fb0c12472d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:40.255378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:40.266537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf708f2-d480-48aa-924b-11fb0c12472d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:40.304657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:40.305344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:40.305622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:40.305892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:40.306494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:40.307909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:40.317677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:40.318754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:40.319592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 501/1017] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:40.325039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:40.402138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:40.428085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355087b7-6ea4-4206-b0c4-715ccac61abf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:40.462871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:40.463469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:40.463751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:40.464010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:40.468321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:40.468321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:40.468321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:40.475489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 517/1018] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:40.481701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:40.590089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2ba86-4395-40cf-be58-33b4941411f1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:40.627453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:40.628048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:40.628319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:40.628581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:40.629229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:40.629526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:40.630878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e7ee1-4423-4a94-bb10-1e085d1987b5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:40.641472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 502/1019] 10.222.0.85 () {56 vars in 1112 bytes} [Wed Nov 2 12:16:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:40.643062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:40.745505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93630626-d48f-44bc-9931-ef56cc4464d3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:40.780840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717987a-fb37-476b-9bb5-6edbd6b8799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:40.781483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717987a-fb37-476b-9bb5-6edbd6b8799b None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:40.781762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717987a-fb37-476b-9bb5-6edbd6b8799b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:40.782029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717987a-fb37-476b-9bb5-6edbd6b8799b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:40.782707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717987a-fb37-476b-9bb5-6edbd6b8799b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:40.784068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717987a-fb37-476b-9bb5-6edbd6b8799b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:40.793765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:40.813046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 518/1020] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:16:41.552163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd545-411e-47f3-b8bb-5a74cdd7d878 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:41.552682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd545-411e-47f3-b8bb-5a74cdd7d878 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:41.552760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd545-411e-47f3-b8bb-5a74cdd7d878 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:41.554245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 503/1021] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:41.559545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:41.559871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:41.560112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:41.570067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacea2cb-45bc-4a72-aae2-d131297646df None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:41.570412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacea2cb-45bc-4a72-aae2-d131297646df None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:41.570618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacea2cb-45bc-4a72-aae2-d131297646df None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:41.571982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 504/1022] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:41.578047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:41.578811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:41.579644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:41.580110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:41.581354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:41.605294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:41.606836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:41.626825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:41.668156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 519/1023] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:41.693046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 505/1024] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:41.937208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:41.961961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:42.054818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b7a68-52f4-4de3-a799-7f394d9e6485 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.075193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9da756-2b91-4e6f-94a8-6563b73d1f4d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.082854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:42.097631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.098634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.098634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.098869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.099460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.100805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:42.109724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:42.112691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:42.113899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 520/1025] 10.222.0.85 () {56 vars in 1105 bytes} [Wed Nov 2 12:16:41 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:42.117264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:42.131959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.132603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.132884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.133461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.134690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.136201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.137832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 506/1026] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:42.143938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:42.176274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:42.253290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70110445-1f98-41b9-97b6-d0e95f3e2d9b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.282239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7e8ed-d4b8-4cef-a1dd-4f3ebe3fd33b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.295403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.296017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.296295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.296567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.297349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] RBAC: Adding query filter params (name=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:42.298815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.299138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.310550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 521/1027] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:42.312410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:42.327948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.328561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.328835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.329141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.329807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.331246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.359093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 507/1028] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:16:42 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:16:42.389693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:42.459887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e6b91-b4a3-4116-816c-5244588bc055 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.503447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.504117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.504398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.504664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.505354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] RBAC: Adding query filter params (type=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:42.505658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.507073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.513868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 522/1029] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:42.520627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:42.617501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:42.644732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcf1e0-fc81-4865-b497-83a8b59859e8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.681315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.681936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.682214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.682486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.683194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] RBAC: Adding query filter params (interface=public, service_id=49a448a367044cfc8c29bdaacfb011a5, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:42.683500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.684886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f0161-bef3-449b-bff6-c92c204a7a96 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.692510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 508/1030] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:16:42 2022] GET /identity/v3/endpoints?service_id=49a448a367044cfc8c29bdaacfb011a5&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:42.698119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:42.804522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412272-c680-48d2-8051-0e2e96e3f922 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:42.819603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:42.842015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc9bd6-b484-4e35-a248-a325489df281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:42.842661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc9bd6-b484-4e35-a248-a325489df281 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:42.842939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc9bd6-b484-4e35-a248-a325489df281 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:42.843240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc9bd6-b484-4e35-a248-a325489df281 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:42.843821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc9bd6-b484-4e35-a248-a325489df281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:42.845240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc9bd6-b484-4e35-a248-a325489df281 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:42.852266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 523/1031] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:16:42 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:16:43.124231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f4cdf-bade-42a2-afa8-cfc3f61b5e60 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:43.124417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f4cdf-bade-42a2-afa8-cfc3f61b5e60 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:43.124690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f4cdf-bade-42a2-afa8-cfc3f61b5e60 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:43.126091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 509/1032] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:43.131660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:43.132187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:43.132187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:43.151879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:43.153127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:43.225366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 524/1033] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:43.492532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f4cdf-bade-42a2-afa8-cfc3f61b5e60 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:43.605943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f4cdf-bade-42a2-afa8-cfc3f61b5e60 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:43.641474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:43.642136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:43.642396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:43.642631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:43.650921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:43.656674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:43.661552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:43.662498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:43.663432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 510/1034] 10.222.0.85 () {56 vars in 1120 bytes} [Wed Nov 2 12:16:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:43.668855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:43.779845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8fa-8db1-4834-8550-9d6dcebcae6f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:43.790558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:43.817933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:43.818537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:43.818808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:43.819074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:43.819728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:43.820022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:43.821407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:43.834568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 525/1035] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:16:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:43.835853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:43.947474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6fbe0-43ea-4c30-ad58-35dfe05673bc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:43.984820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:43.985485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:43.985764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:43.986034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:43.997209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:43.999134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:44.013836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:44.014845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:44.015831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:44.015831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:44.015831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:44.017075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 511/1036] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:44.020818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:44.126753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118943ce-b741-4ce2-b80e-2b9d32a5c508 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:44.161778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:44.162416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:44.162660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:44.162907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:44.163551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:44.163850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:44.165237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:44.183244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 526/1037] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:44.184525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:44.242812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:44.295108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3b2b-214b-4d8f-963e-504dd5578659 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:44.335689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:44.336576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:44.337017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:44.337454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:44.346702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:44.348383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:44.361112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:44.362214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:44.363458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 512/1038] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:16:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:44.368788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:44.400831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:44.480058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9f04-b7b5-4cae-a44a-28924cb4ce18 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:44.517621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:44.518226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:44.518498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:44.518763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:44.519495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:44.519795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:44.521228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa7c04-0e6b-4365-958d-5ee6d77834f7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:44.546153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 527/1039] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:16:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:44.547798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:44.663971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d0beb-0ed7-4044-9a11-55d78ac10ee5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:44.664219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d0beb-0ed7-4044-9a11-55d78ac10ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:44.664443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b9ed-e09f-4877-9b5f-17962b2afb8b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:44.664870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d0beb-0ed7-4044-9a11-55d78ac10ee5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:44.666240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 528/1040] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:44.671555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:44.671827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:44.672097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:44.685900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:44.694083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:44.695910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:44.710170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15f7f4-cda9-477b-849e-ef8340d6f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:44.710780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f7f4-cda9-477b-849e-ef8340d6f146 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:44.711055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f7f4-cda9-477b-849e-ef8340d6f146 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:44.711322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f7f4-cda9-477b-849e-ef8340d6f146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:44.773221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 529/1041] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:44.799033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15f7f4-cda9-477b-849e-ef8340d6f146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:44.800511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15f7f4-cda9-477b-849e-ef8340d6f146 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:44.933566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 513/1042] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:44.940025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:44.963712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.074490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1cce7-c362-4913-9fa4-3da409799c15 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.086021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816edf64-846a-4b39-9647-a16424c2620a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.086367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816edf64-846a-4b39-9647-a16424c2620a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.086646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816edf64-846a-4b39-9647-a16424c2620a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.088131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 514/1043] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:45.094176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.094454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.094721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.112985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:45.114226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:45.116592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.117256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.117528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.117792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.122718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.128968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.130433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:45.148059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:45.150312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:45.152205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:45.152205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:45.152205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:45.152205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 530/1044] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:45.152205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.209690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 515/1045] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:45.210836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.233878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.275082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2384a1-f03a-4d4b-b036-cf03d8fa9619 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.315606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.316282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.316557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.316818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.329631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:45.334856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:45.336229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:45.337350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 531/1046] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:45.341941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.359610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554ddf3-87f7-4a9e-9119-a489b6d0d052 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.404344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.405593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.405921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.406190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.406848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:45.407145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.408616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.429920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 516/1047] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:45.431160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.435304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.483202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2688f-4828-487c-80bd-1d0dd8245d00 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.523892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.531384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.532004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.532279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.532546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.533252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:45.533553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.534914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.551794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 532/1048] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:45.551794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.563985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e6b8d-253b-4f4c-8b3d-c671505ab424 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.613620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.614293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.614615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.614987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.622338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.624325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:45.637521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:45.638739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 517/1049] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:45.640228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.696238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cff5cf-59b6-49db-a889-1737c7835466 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.709803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.731871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.732434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.732707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.732989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.733601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.735092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:45.744275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:45.745545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:45.746411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 533/1050] 10.222.0.85 () {56 vars in 1105 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:45.748986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.761163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80be9-4f9a-4432-b5ef-886050e867eb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.798256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.798894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.799159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.799427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] PATH_INFO: `/v3/users/4cdd42af07764fcda2565835cf1d6724` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.815373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdd42af07764fcda2565835cf1d6724)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.817687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.819757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:45.825221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 518/1051] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/users/4cdd42af07764fcda2565835cf1d6724 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:45.825221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.875422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d019d-c1b1-495a-a4b3-039496cf9b12 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.917471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:45.918096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:45.918378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:45.918652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:45.919372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:45.919686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:45.921260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:45.937283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 534/1052] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:45.938170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:45.949214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb2a5f-87ee-4d1c-a52e-27512ce30ca6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:45.976516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:46.006169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.006988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.007365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.007879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.009592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] RBAC: Adding query filter params (name=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:46.009978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.012497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.019578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 519/1053] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:46.020515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:46.081184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b9de-7e16-43e8-aaa8-cdb68ef368a0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:46.102523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:46.123787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.124905 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.125204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.125473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.163165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939870d-4cd0-4f25-85ae-5b5fdc236d1c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:46.205759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.206447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.206744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.207053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.207975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] RBAC: Adding query filter params (role.id=2e3744c807604e12b85f8c897ed8316f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=c173404c311b43e29193004dc41480b8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:46.208410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.210418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.210651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.211938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.218127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 520/1054] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:46 2022] GET /identity/v3/role_assignments?user.id=c173404c311b43e29193004dc41480b8&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=2e3744c807604e12b85f8c897ed8316f => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:46.219056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:46.290900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:46.356111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 535/1055] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:46.363061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:46.367999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19412a4e-be92-451f-b33a-5b24833b1516 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:46.393173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:46.418044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.421064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.421064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.421064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.421064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] RBAC: Adding query filter params (type=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:46.421064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.422143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.429817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 521/1056] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:46.437705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:46.515662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137933c-101e-4b03-872f-45526a7a032d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:46.561899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.562845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.563136 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.563413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.564811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] RBAC: Authorizing `identity:get_project(project_id=3fcef62b54f644e8a69ccffc2c32d9b9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.566446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.568291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5920eb7-7592-4e56-8ba4-d38cb862240a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:46.568435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 536/1057] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:16:46 2022] GET /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9 => generated 311 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:46.574343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:46.585893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:46.613849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3659b58-e023-45ea-b797-f246e2aaa585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.614544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3659b58-e023-45ea-b797-f246e2aaa585 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.614830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3659b58-e023-45ea-b797-f246e2aaa585 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.615097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3659b58-e023-45ea-b797-f246e2aaa585 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.615849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3659b58-e023-45ea-b797-f246e2aaa585 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.617306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3659b58-e023-45ea-b797-f246e2aaa585 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.639723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 522/1058] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:16:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:16:46.702877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:46.727540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abdf75-881c-4a1a-8972-233bbaf86a49 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:46.813874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:46.817999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:46.817999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:46.817999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:46.817999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9, user.id=4cdd42af07764fcda2565835cf1d6724) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:46.817999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:46.818759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9196e44-63f9-4a20-9bdf-14ce6cde6746 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:46.833309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 537/1059] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:46 2022] GET /identity/v3/role_assignments?user.id=4cdd42af07764fcda2565835cf1d6724&scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:48.584519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e1ce2-f893-4b92-b1de-495f9a20289f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:48.585445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e1ce2-f893-4b92-b1de-495f9a20289f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:48.585927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e1ce2-f893-4b92-b1de-495f9a20289f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:48.587396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 523/1060] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:48.594456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a00c-df06-41eb-a88e-103773f1dcff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:48.594942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a00c-df06-41eb-a88e-103773f1dcff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:48.595148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a00c-df06-41eb-a88e-103773f1dcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:48.614594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0996a00c-df06-41eb-a88e-103773f1dcff None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:48.615814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0996a00c-df06-41eb-a88e-103773f1dcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:48.698127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 538/1061] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:48.956332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779676f4-dfbe-4bfd-a990-4b2927d6ddce None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:48.956729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779676f4-dfbe-4bfd-a990-4b2927d6ddce None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:48.956911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779676f4-dfbe-4bfd-a990-4b2927d6ddce None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:48.958540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 524/1062] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:48.964579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d35a73-4d7f-41af-a18e-66735a641a8a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:48.964894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d35a73-4d7f-41af-a18e-66735a641a8a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:48.965284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d35a73-4d7f-41af-a18e-66735a641a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:48.969119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779676f4-dfbe-4bfd-a990-4b2927d6ddce None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:48.985939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f5d35a73-4d7f-41af-a18e-66735a641a8a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:48.987172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5d35a73-4d7f-41af-a18e-66735a641a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:49.065066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 539/1063] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:49.076231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da039865-4af3-4f4a-89eb-735b6040f329 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.076532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da039865-4af3-4f4a-89eb-735b6040f329 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.077344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da039865-4af3-4f4a-89eb-735b6040f329 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.078668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 540/1064] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:49.083617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.083896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.084166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.085130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779676f4-dfbe-4bfd-a990-4b2927d6ddce None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.091402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.104764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:49.106002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:49.121686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.122318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.122600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.122894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.134825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:49.140183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:49.141346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:49.142232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 525/1065] 10.222.0.85 () {56 vars in 1120 bytes} [Wed Nov 2 12:16:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:49.147146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.179897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.183636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 541/1066] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:49.255719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097b69f-ca16-4835-b529-8b3871552d8e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.292017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.292642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.292915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.293202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.293890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:49.294184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:49.295694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:49.310463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 526/1067] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:49.312826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.340148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.435308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058a022-fc5e-4447-983e-f7ae8360fe88 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.455429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.466336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27027-6252-4ddd-9db2-c6dee2b636c8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.478462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.479105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.479383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.479654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.481830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.491775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:49.493266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:49.506849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.507800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.508295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.509067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:49.509588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:49.510581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:49.511540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:49.511540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:49.511540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:49.511540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 542/1068] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:49.511781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.607171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:49.609226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:49.625497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d4d29-e4f6-440e-8fff-4d7cb08d07fa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.660031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.660657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.660936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.661237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.673080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:49.677884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:49.678883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:49.679833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 543/1069] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:49.680549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.703803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.714968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 527/1070] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 12:16:49.716157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.733147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.788056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427fac6-55d5-4a93-8faf-cb3d16f6c4a9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.820106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb4d56-0e1d-4f39-ba48-e1a99737ab73 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.822420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.823033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.823302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.823571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.824209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:49.824506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:49.825890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:49.843963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 544/1071] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:49.843963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.857843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.858500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.858793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.859073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:49.859789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:49.860084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:49.861623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:49.870413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 528/1072] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:49.871291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:49.926546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.947064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae2b1-e4d5-4c90-8c1e-6d26c2e337bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.956960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:49.975046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce999de-c706-4fe8-9a54-d4d5599972bc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:49.988520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:49.989217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:49.989512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:49.989779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.000561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.002225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.009220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.009849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.010119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.010383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:50.018836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:50.020032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:50.020945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:50.020945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:50.020945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:50.021157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.021247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 545/1073] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:50.021329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.021400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:50.034555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:50.035642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 529/1074] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:16:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:50.036638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.135584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320d93e-4173-4be7-8d11-fc5bfec94fcd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.144896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220927dd-df9f-4695-9639-41cf77afc096 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.158038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:50.173272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.173912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.174194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.174469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] PATH_INFO: `/v3/users/4cdd42af07764fcda2565835cf1d6724` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.175617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdd42af07764fcda2565835cf1d6724)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.177062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.178655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 546/1075] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/users/4cdd42af07764fcda2565835cf1d6724 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:50.179409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.180225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.180895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.181252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.181576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.182280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:50.182634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.184071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.209733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 530/1076] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:50.209733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.210049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:50.300880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2560df9-32ec-4fab-ae05-769aa4446259 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.322280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebad16-26f4-41f0-aa5b-15a08b774a57 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.341466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.342099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.342413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.342692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.343447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:50.343757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.345260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.361494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.362164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.362442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.362704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.363955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] RBAC: Authorizing `identity:get_project(project_id=3fcef62b54f644e8a69ccffc2c32d9b9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.365448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.371354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 547/1077] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:50.371581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 531/1078] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:50.373160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.373279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.423905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:50.495133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:50.500401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bb9d0-8f7b-46ca-83c0-e15c0fa4eea4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.519472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060fed-3880-4ca2-bf3d-0ebe11e65608 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.549794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.550477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.550797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.551072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.562886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.564389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.570342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.571009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.571286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.571565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9/users/4cdd42af07764fcda2565835cf1d6724/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.573801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fcef62b54f644e8a69ccffc2c32d9b9, user_id=4cdd42af07764fcda2565835cf1d6724, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.575169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:50.580734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:50.582224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:50.583535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:50.583535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:50.583535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:50.583535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 532/1079] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:50.590732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.595465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 548/1080] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:16:50 2022] PUT /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9/users/4cdd42af07764fcda2565835cf1d6724/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:16:50.739197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:50.750098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a1f34-3058-4a82-b9b1-9abab3e879b9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.794463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.795074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.795351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.795648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.796294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:50.796599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.798046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.812157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 533/1081] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:50.819697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:50.933172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bc3f5-bbba-4343-847b-de6667fb6129 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:50.971247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:50.971883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:50.972159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:50.972428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:50.979958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:50.981480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:50.995028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:50.996113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:50.997567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 549/1082] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:51.002776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:51.114473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7835db-4370-4b8e-966d-997254f947db None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:51.157537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:51.158183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:51.158454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:51.158717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:51.159418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:51.159718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:51.161234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c5116-d94a-49cf-a0f3-bf2d0cc45237 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:51.175273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 534/1083] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:51.182100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:51.186217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:51.292616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693c862-66fd-4a81-80e3-076e923e59f4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:51.329635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1560f-d76d-43eb-b87f-0a5f00f9fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:51.330308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1560f-d76d-43eb-b87f-0a5f00f9fce6 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:51.330579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1560f-d76d-43eb-b87f-0a5f00f9fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:51.330842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1560f-d76d-43eb-b87f-0a5f00f9fce6 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/c173404c311b43e29193004dc41480b8/roles/2e3744c807604e12b85f8c897ed8316f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:51.333167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1560f-d76d-43eb-b87f-0a5f00f9fce6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=c173404c311b43e29193004dc41480b8, role_id=2e3744c807604e12b85f8c897ed8316f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:51.334629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1560f-d76d-43eb-b87f-0a5f00f9fce6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:51.349636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 550/1084] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:16:51 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/c173404c311b43e29193004dc41480b8/roles/2e3744c807604e12b85f8c897ed8316f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:16:52.800820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae05dc-669c-4fc3-b5ef-949c8c5e65d0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:52.801390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae05dc-669c-4fc3-b5ef-949c8c5e65d0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:52.801390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae05dc-669c-4fc3-b5ef-949c8c5e65d0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:52.802989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 535/1085] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:52.810990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:52.812253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:52.812253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:52.832599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:52.833858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:52.903125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:52.913595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 551/1086] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:53.185578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae05dc-669c-4fc3-b5ef-949c8c5e65d0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:53.201067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:53.289620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae05dc-669c-4fc3-b5ef-949c8c5e65d0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:53.323780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:53.324429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:53.324703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:53.324973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:53.335822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:53.340926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:53.342081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:53.343588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 536/1087] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:53.349020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:53.451293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9da08b-2741-4ecc-b9c7-948ca8be53af None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:53.494233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:53.494848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:53.495125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:53.495395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:53.496052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:53.496352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:53.497742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be382-b23a-42e7-9c2e-8da6c3f70607 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:53.506654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 552/1088] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:53.512844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:53.557804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027b2c-0bd2-4335-a6bd-c4fdf2e3ab21 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:53.558370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027b2c-0bd2-4335-a6bd-c4fdf2e3ab21 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:53.558676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027b2c-0bd2-4335-a6bd-c4fdf2e3ab21 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:53.560434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 553/1089] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:16:53.566322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:53.566601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:53.566876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:53.582971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:53.585730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:53.586926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:53.617937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9370-c672-4434-ac23-2617bcde4750 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:53.645162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:53.654554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:53.655205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:53.655477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:53.655750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] PATH_INFO: `/v3/users/4cdd42af07764fcda2565835cf1d6724` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:53.657049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] RBAC: Authorizing `identity:get_user(user_id=4cdd42af07764fcda2565835cf1d6724)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:53.658482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:53.660183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 537/1090] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:16:53 2022] GET /identity/v3/users/4cdd42af07764fcda2565835cf1d6724 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:53.663562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 554/1091] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:16:53.666086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:53.769359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651632d-b6aa-47c7-a191-a0e2f99b35bf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:53.802388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:53.803032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:53.803236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:53.803503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:53.814588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:53.870418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:53.871959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:53.932245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:53.971092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 538/1092] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 12:16:53.976724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.019179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:54.033924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6a99-f3a8-415e-98f0-bd47b2099e44 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.068885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.069539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.069817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.070085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.077510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad66066-3d21-40c1-99a2-dd2c9c5ed471 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.080514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:16:54.085336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:54.086516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:54.087601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 555/1093] 10.222.0.85 () {56 vars in 1120 bytes} [Wed Nov 2 12:16:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:54.092309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.114568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.115187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.115458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.115723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.116839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] RBAC: Authorizing `identity:get_project(project_id=3fcef62b54f644e8a69ccffc2c32d9b9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.118293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.119904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 539/1094] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:16:53 2022] GET /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:54.126421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.142083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:54.194925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e3c95-8230-49a0-8f33-c80a515a4c00 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.225992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081b930-6621-4423-b90b-ab31b4e66428 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.231129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.231731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.232008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.232278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.232937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:54.233275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.234628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.235318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:54.242991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 556/1095] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:54.248734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.260656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.261310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.261586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.261855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.262582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9, user.id=4cdd42af07764fcda2565835cf1d6724) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:54.262870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.264793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.270802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 540/1096] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/role_assignments?user.id=4cdd42af07764fcda2565835cf1d6724&scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:16:54.353872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f6ee-6dd2-467e-be59-a37c1be86a19 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.363642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:54.390319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745fd44-458e-491e-83ce-463476837941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.390959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745fd44-458e-491e-83ce-463476837941 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.391232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745fd44-458e-491e-83ce-463476837941 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.391497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745fd44-458e-491e-83ce-463476837941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.403537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745fd44-458e-491e-83ce-463476837941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.405083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745fd44-458e-491e-83ce-463476837941 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f745fd44-458e-491e-83ce-463476837941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:54.420836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:54.421954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:54.422888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:54.422888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:54.422888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:54.422888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 557/1097] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:16:54.428401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.539597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33df93-63e8-4640-a37b-a092384fbb06 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.573494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.574092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.574365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.574630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.575270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:54.575564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.576975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.591803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 541/1098] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:54.598618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745fd44-458e-491e-83ce-463476837941 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.698883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745fd44-458e-491e-83ce-463476837941 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.732162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.732778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.733106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.733376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.740865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.742364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:16:54.754123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:54.755249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:16:54.756244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 558/1099] 10.222.0.85 () {56 vars in 1093 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:54.761287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.792916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:54.862476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b867f-87c9-4ed4-801a-d245cc5b2c29 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:54.896475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:54.897096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:54.897374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:54.897640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:54.898561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:54.898865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:54.900281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:54.921206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 542/1100] 10.222.0.85 () {56 vars in 1133 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:54.926951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:54.949340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:55.026727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d7e6-7d1a-4d14-9c49-6a7e329949d5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:55.060923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:55.062112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:55.062390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:55.062659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:55.129125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:55.130565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:55.150863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:55.236631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 559/1101] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:16:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:16:55.242302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:55.331730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:55.341795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794dab9-0582-4095-b001-1390567b9de9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:55.376168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:55.376789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:55.377093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:55.377364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:55.387745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:55.389138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:16:55.404403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:55.406127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:16:55.407360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:16:55.407360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:16:55.407360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:55.409150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 543/1102] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:55.411646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:55.510997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d8d8-9790-4b3d-b00a-064088952985 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:55.531322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:55.545724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:55.546830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:55.547106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:55.547375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:55.548009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:55.548374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:55.549681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:55.560934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 560/1103] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:55.566644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:55.694794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d45b92-4b90-456a-95fa-69d9d9f1b661 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:55.729127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:55.729754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:55.730034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:55.730308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:55.737352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:55.738788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:16:55.750723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:16:55.751654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:16:55.752578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 544/1104] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:16:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:16:55.757779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:55.859831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236e541-8a24-4259-affd-5a1b568bdb68 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:55.898775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:55.899572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:55.899852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:55.900119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:55.900813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:55.901145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:55.902570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bad11-d2c0-48e2-858e-f77596f324af None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:55.915818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 561/1105] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:16:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:55.923266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:55.926331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:56.034162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9cb0-6a0b-43e7-85be-67866e2407db None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:56.069397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:56.070016 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:56.070294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:56.070562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:56.071294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] RBAC: Adding query filter params (role.id=2e3744c807604e12b85f8c897ed8316f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=c173404c311b43e29193004dc41480b8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:16:56.071601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:56.073057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62405b-117b-41c8-ba84-5885c04f22ad None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:56.080347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 545/1106] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:16:55 2022] GET /identity/v3/role_assignments?user.id=c173404c311b43e29193004dc41480b8&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=2e3744c807604e12b85f8c897ed8316f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:56.479750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7a182-b87d-4c60-b823-1b46dcc165da None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:56.480240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7a182-b87d-4c60-b823-1b46dcc165da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:56.480349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7a182-b87d-4c60-b823-1b46dcc165da None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:56.481671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 562/1107] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:56.487831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:56.488111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:56.488386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:56.508017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:56.509556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:56.572148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:56.582942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 546/1108] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:56.854763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7a182-b87d-4c60-b823-1b46dcc165da None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:56.902713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:56.963671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7a182-b87d-4c60-b823-1b46dcc165da None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:56.998327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd81359-4c1b-4924-8c6a-c18f81bdd971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:56.998982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd81359-4c1b-4924-8c6a-c18f81bdd971 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:56.999259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd81359-4c1b-4924-8c6a-c18f81bdd971 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:56.999529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd81359-4c1b-4924-8c6a-c18f81bdd971 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:57.000645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd81359-4c1b-4924-8c6a-c18f81bdd971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:57.002104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd81359-4c1b-4924-8c6a-c18f81bdd971 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:57.003670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 563/1109] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:16:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:57.009727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:57.105417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e5071-371b-4387-946f-dee59789d0ee None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:57.138665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7d05b-610a-41c3-8835-78fdd9645126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:57.139262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7d05b-610a-41c3-8835-78fdd9645126 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:57.140090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7d05b-610a-41c3-8835-78fdd9645126 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:57.140350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7d05b-610a-41c3-8835-78fdd9645126 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:57.141021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7d05b-610a-41c3-8835-78fdd9645126 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:57.142446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7d05b-610a-41c3-8835-78fdd9645126 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:57.163876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 547/1110] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:16:57 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:16:59.361316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa45a0-7a86-47dd-b2f6-a9c65c1b02b6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:59.361897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa45a0-7a86-47dd-b2f6-a9c65c1b02b6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:59.361897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa45a0-7a86-47dd-b2f6-a9c65c1b02b6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:59.363174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 564/1111] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:16:59.368504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:59.368949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:59.369108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:59.387919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:16:59.389165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:16:59.437363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:59.458620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 548/1112] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:16:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:16:59.733630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa45a0-7a86-47dd-b2f6-a9c65c1b02b6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:59.833171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa45a0-7a86-47dd-b2f6-a9c65c1b02b6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:16:59.842418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:16:59.867600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:16:59.868225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:16:59.868503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:16:59.868774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] PATH_INFO: `/v3/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:16:59.870196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] RBAC: Authorizing `identity:get_role(role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:16:59.871551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:16:59.872968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 565/1113] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:16:59 2022] GET /identity/v3/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:16:59.879545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:16:59.983841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446db657-0ed4-434b-b57a-6c67582b2620 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:00.019606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:00.020231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:00.020499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:00.020765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] PATH_INFO: `/v3/users/6490b50659334b65b8a8fc171fce2e7f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:00.034011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] RBAC: Authorizing `identity:get_user(user_id=6490b50659334b65b8a8fc171fce2e7f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:00.035475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:00.037116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 549/1114] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:16:59 2022] GET /identity/v3/users/6490b50659334b65b8a8fc171fce2e7f => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:17:00.042977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:00.141975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc96a2-9150-4944-98e8-0c9aaced209a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:00.174573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:00.175184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:00.175510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:00.175784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:00.220320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:00.243518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:00.244882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:00.341915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 566/1115] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:17:00.347917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:00.390955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:00.448598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea8b01-7b57-4324-85a8-d45db7c862cc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:00.482570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bfc07-fbcb-480f-8937-f6a641f34e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:00.483222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bfc07-fbcb-480f-8937-f6a641f34e3c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:00.483501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bfc07-fbcb-480f-8937-f6a641f34e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:00.483769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bfc07-fbcb-480f-8937-f6a641f34e3c None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:00.484890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bfc07-fbcb-480f-8937-f6a641f34e3c None admin] RBAC: Authorizing `identity:get_project(project_id=3fcef62b54f644e8a69ccffc2c32d9b9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:00.486351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bfc07-fbcb-480f-8937-f6a641f34e3c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:00.487981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 550/1116] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:00 2022] GET /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:00.494184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:00.568568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:00.591418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628301c1-bc24-4642-88b1-f1410616bae1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:00.623666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:00.624759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:00.625057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:00.625325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:00.626028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] RBAC: Adding query filter params (role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9, user.id=6490b50659334b65b8a8fc171fce2e7f) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:00.626324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:00.627708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f09354-1461-417c-af56-7d0df4318262 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:00.632801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 567/1117] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:17:00 2022] GET /identity/v3/role_assignments?user.id=6490b50659334b65b8a8fc171fce2e7f&scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:02.786054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41eb4-7406-4692-87a1-fc091cbddac2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:02.786576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41eb4-7406-4692-87a1-fc091cbddac2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:02.786576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41eb4-7406-4692-87a1-fc091cbddac2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:02.787865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 551/1118] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:02.793111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:02.793495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:02.793654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:02.811872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:02.813093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:02.878455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 568/1119] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:03.151232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41eb4-7406-4692-87a1-fc091cbddac2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:03.250730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41eb4-7406-4692-87a1-fc091cbddac2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:03.260070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:03.284653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:03.285315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:03.285587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:03.285852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] PATH_INFO: `/v3/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:03.287236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] RBAC: Authorizing `identity:get_role(role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:03.288601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:03.289949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 552/1120] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:03 2022] GET /identity/v3/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:03.296380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:03.362139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:03.393406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fba19-55cb-413e-82fd-aa59a1bccc00 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:03.426627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:03.427175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:03.427449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:03.427715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] PATH_INFO: `/v3/users/6490b50659334b65b8a8fc171fce2e7f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:03.428822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] RBAC: Authorizing `identity:get_user(user_id=6490b50659334b65b8a8fc171fce2e7f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:03.430262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:03.431825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 569/1121] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:03 2022] GET /identity/v3/users/6490b50659334b65b8a8fc171fce2e7f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:03.437900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:03.536370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfe94c-16d9-49fc-90ca-fc2d3ef0a6a8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:03.569200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1b395-32b2-4c48-89be-1d633763db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:03.569810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1b395-32b2-4c48-89be-1d633763db38 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:03.570081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1b395-32b2-4c48-89be-1d633763db38 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:03.570348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1b395-32b2-4c48-89be-1d633763db38 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:03.571439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1b395-32b2-4c48-89be-1d633763db38 None admin] RBAC: Authorizing `identity:get_project(project_id=3fcef62b54f644e8a69ccffc2c32d9b9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:03.572862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1b395-32b2-4c48-89be-1d633763db38 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:03.574498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 553/1122] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:03 2022] GET /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:03.581231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:03.677993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431a312-de08-46e5-a8d8-0aff915a4172 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:03.711733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eaa44-8d61-448d-b724-54e1c22086c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:03.712382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eaa44-8d61-448d-b724-54e1c22086c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:03.712661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eaa44-8d61-448d-b724-54e1c22086c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:03.712934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eaa44-8d61-448d-b724-54e1c22086c1 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9/users/6490b50659334b65b8a8fc171fce2e7f/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:03.715102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eaa44-8d61-448d-b724-54e1c22086c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fcef62b54f644e8a69ccffc2c32d9b9, user_id=6490b50659334b65b8a8fc171fce2e7f, role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:03.716527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37eaa44-8d61-448d-b724-54e1c22086c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:03.725378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:03.731258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 570/1123] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:17:03 2022] PUT /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9/users/6490b50659334b65b8a8fc171fce2e7f/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:17:05.876851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221e27-eaae-4ada-812c-2c6d3e7c7ac1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:05.877510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221e27-eaae-4ada-812c-2c6d3e7c7ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:05.877510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221e27-eaae-4ada-812c-2c6d3e7c7ac1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:05.878681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 554/1124] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:05.884111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:05.884756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:05.884906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:05.902846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:05.904214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:05.970887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 571/1125] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:06.243554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221e27-eaae-4ada-812c-2c6d3e7c7ac1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:06.291306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:06.345023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221e27-eaae-4ada-812c-2c6d3e7c7ac1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:06.377683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:06.378308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:06.378584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:06.378855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] PATH_INFO: `/v3/roles/851b9a649ef74da4ad1130c9c3fa4812` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:06.380232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] RBAC: Authorizing `identity:get_role(role_id=851b9a649ef74da4ad1130c9c3fa4812)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:06.381619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:06.382937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 555/1126] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:06 2022] GET /identity/v3/roles/851b9a649ef74da4ad1130c9c3fa4812 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:06.389280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:06.488279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e4d7-6411-4357-9668-6461c1ebe99e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:06.513803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:06.522315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:06.522938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:06.523216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:06.523482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] PATH_INFO: `/v3/users/6490b50659334b65b8a8fc171fce2e7f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:06.524597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] RBAC: Authorizing `identity:get_user(user_id=6490b50659334b65b8a8fc171fce2e7f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:06.526039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:06.527607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 572/1127] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:06 2022] GET /identity/v3/users/6490b50659334b65b8a8fc171fce2e7f => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:06.534091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:06.631895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3ca8-b5b4-4d29-bcce-b5366a8ce281 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:06.646192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:06.666823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:06.667414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:06.667690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:06.667953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:06.733180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:06.734643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:06.832314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 556/1128] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:17:06.838304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:06.934049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf622d-a20e-44c0-8c31-5e9d55df345b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:06.967323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffb34a-68d2-4d6d-b03f-bbf991e9d3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:06.967955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffb34a-68d2-4d6d-b03f-bbf991e9d3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:06.968227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffb34a-68d2-4d6d-b03f-bbf991e9d3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:06.968492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffb34a-68d2-4d6d-b03f-bbf991e9d3f0 None admin] PATH_INFO: `/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:06.969668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffb34a-68d2-4d6d-b03f-bbf991e9d3f0 None admin] RBAC: Authorizing `identity:get_project(project_id=3fcef62b54f644e8a69ccffc2c32d9b9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:06.971059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffb34a-68d2-4d6d-b03f-bbf991e9d3f0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:06.972728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 573/1129] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:06 2022] GET /identity/v3/projects/3fcef62b54f644e8a69ccffc2c32d9b9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:06.978957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:06.993274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:07.076776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189967f-572e-40bc-9a9d-dc0b5602cc46 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:07.110144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:07.110666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:07.110941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:07.111206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:07.111920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] RBAC: Adding query filter params (role.id=851b9a649ef74da4ad1130c9c3fa4812, scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9, user.id=6490b50659334b65b8a8fc171fce2e7f) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:07.112222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:07.113670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7986792-c8f8-40ad-89b4-af6389151585 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:07.119444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 557/1130] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:17:06 2022] GET /identity/v3/role_assignments?user.id=6490b50659334b65b8a8fc171fce2e7f&scope.project.id=3fcef62b54f644e8a69ccffc2c32d9b9&role.id=851b9a649ef74da4ad1130c9c3fa4812 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:09.298007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63ee9-eb17-4263-b8fa-cd761a0196db None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:09.298543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63ee9-eb17-4263-b8fa-cd761a0196db None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:09.298543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63ee9-eb17-4263-b8fa-cd761a0196db None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:09.299863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 574/1131] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:09.305217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:09.305611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:09.305762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:09.324312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:09.325574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:09.391176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 558/1132] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:09.657954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63ee9-eb17-4263-b8fa-cd761a0196db None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:09.722916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:09.763468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63ee9-eb17-4263-b8fa-cd761a0196db None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:09.799659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357629e-dac7-4cf8-ae9d-966d5a9a404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:09.800296 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357629e-dac7-4cf8-ae9d-966d5a9a404b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:09.800574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357629e-dac7-4cf8-ae9d-966d5a9a404b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:09.800846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357629e-dac7-4cf8-ae9d-966d5a9a404b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:09.802045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357629e-dac7-4cf8-ae9d-966d5a9a404b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:09.803574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357629e-dac7-4cf8-ae9d-966d5a9a404b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:09.805210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 575/1133] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:17:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:09.811221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:09.820265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:09.910755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc1bb4-e63c-49d0-85bd-d9ebb91fcc81 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:09.944674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdd168-e911-47ba-84dc-1f0a0281e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:09.945301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdd168-e911-47ba-84dc-1f0a0281e097 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:09.945580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdd168-e911-47ba-84dc-1f0a0281e097 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:09.945849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdd168-e911-47ba-84dc-1f0a0281e097 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:09.946499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdd168-e911-47ba-84dc-1f0a0281e097 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:09.947933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdd168-e911-47ba-84dc-1f0a0281e097 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:09.975757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 559/1134] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:17:09 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:17:12.157718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e7fdf-33bc-4064-961d-65cd41feb33f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:12.158247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e7fdf-33bc-4064-961d-65cd41feb33f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:12.158247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e7fdf-33bc-4064-961d-65cd41feb33f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:12.159582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 576/1135] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:12.164898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:12.165345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:12.165487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:12.184095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:12.185322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:12.237994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:12.252589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 560/1136] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:12.524037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e7fdf-33bc-4064-961d-65cd41feb33f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:12.624465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e7fdf-33bc-4064-961d-65cd41feb33f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:12.653531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:12.659787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991f23b-baf3-4809-b585-5a1f52d80788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:12.660443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991f23b-baf3-4809-b585-5a1f52d80788 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:12.660722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991f23b-baf3-4809-b585-5a1f52d80788 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:12.661011 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991f23b-baf3-4809-b585-5a1f52d80788 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:12.662220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991f23b-baf3-4809-b585-5a1f52d80788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:12.663682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991f23b-baf3-4809-b585-5a1f52d80788 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:12.665372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 577/1137] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:17:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:12.672097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:12.780930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecec6d5-83c4-45ed-95af-bc2683fa4def None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:12.817436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db2cb2-8ae3-4cc9-9899-582f131ddb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:12.818118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db2cb2-8ae3-4cc9-9899-582f131ddb71 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:12.818396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db2cb2-8ae3-4cc9-9899-582f131ddb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:12.818667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db2cb2-8ae3-4cc9-9899-582f131ddb71 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:12.819386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db2cb2-8ae3-4cc9-9899-582f131ddb71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:12.820920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db2cb2-8ae3-4cc9-9899-582f131ddb71 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:12.846969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 561/1138] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:17:12 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:17:15.039466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018c625-eb5c-4db0-9eda-34b48b349a5c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:15.039996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018c625-eb5c-4db0-9eda-34b48b349a5c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:15.039996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018c625-eb5c-4db0-9eda-34b48b349a5c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:15.041359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 578/1139] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:15.047339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:15.047729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:15.047878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:15.066153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:15.067335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:15.106137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:15.134624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 562/1140] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:15.403443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018c625-eb5c-4db0-9eda-34b48b349a5c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:15.504885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018c625-eb5c-4db0-9eda-34b48b349a5c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:15.538808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:15.539423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:15.539696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:15.539960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:15.549980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:17:15.554685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:15.555810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:17:15.556866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 579/1141] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:15.562763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:15.659705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8844e-8f10-4e37-82d5-13cc87fe5fc1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:15.692854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:15.693624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:15.693880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:15.694178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:15.694801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:15.695095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:15.696457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:15.705126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 563/1142] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:15.711154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:15.769974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:15.808880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db264-4e46-4ba9-8083-c1ad05da1551 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:15.843327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:15.843948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:15.844222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:15.844489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] PATH_INFO: `/v3/users/0f7dd171aa464b139d0a3a0c4b441e46` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:15.857585 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] RBAC: Authorizing `identity:get_user(user_id=0f7dd171aa464b139d0a3a0c4b441e46)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:15.859004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:15.860615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 580/1143] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:15 2022] GET /identity/v3/users/0f7dd171aa464b139d0a3a0c4b441e46 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:15.867898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:15.906895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:15.970846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1008-0012-4cc4-8a1f-26e958d5cdda None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:16.004087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:16.004675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:16.004948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:16.005242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:16.069837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:16.071263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:16.102869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:16.166263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 564/1144] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 12:17:16.172880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:16.273666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe69e-063f-4562-ba68-308281fdbd59 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:16.290347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:16.307213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0c548-2110-4dd0-864a-813ff3164942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:16.307833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0c548-2110-4dd0-864a-813ff3164942 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:16.308110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0c548-2110-4dd0-864a-813ff3164942 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:16.308379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0c548-2110-4dd0-864a-813ff3164942 None admin] PATH_INFO: `/v3/projects/53a578f745cc4051a984cf10bbc4c264` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:16.309501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0c548-2110-4dd0-864a-813ff3164942 None admin] RBAC: Authorizing `identity:get_project(project_id=53a578f745cc4051a984cf10bbc4c264)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:16.310908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0c548-2110-4dd0-864a-813ff3164942 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:16.313077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 581/1145] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:16 2022] GET /identity/v3/projects/53a578f745cc4051a984cf10bbc4c264 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:16.319281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:16.415494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbad563-76d2-4641-8241-29f6a93331c7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:16.449555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:16.450148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:16.450422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:16.450690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:16.451406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=53a578f745cc4051a984cf10bbc4c264, user.id=0f7dd171aa464b139d0a3a0c4b441e46) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:16.451709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:16.453182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df5a31-190b-4354-a698-d9e5a088b418 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:16.455979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:16.459363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 565/1146] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:17:16 2022] GET /identity/v3/role_assignments?user.id=0f7dd171aa464b139d0a3a0c4b441e46&scope.project.id=53a578f745cc4051a984cf10bbc4c264&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:18.608914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943589b6-18a8-4e63-87ef-027b4c5f358a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:18.609464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943589b6-18a8-4e63-87ef-027b4c5f358a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:18.609556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943589b6-18a8-4e63-87ef-027b4c5f358a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:18.610823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 582/1147] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:18.616064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:18.616491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:18.616582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:18.635124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:18.636333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:18.706782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 566/1148] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:18.980678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943589b6-18a8-4e63-87ef-027b4c5f358a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:19.085637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943589b6-18a8-4e63-87ef-027b4c5f358a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:19.120782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:19.121430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:19.121704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:19.121975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:19.132190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:17:19.137259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:19.138427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:17:19.140176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 583/1149] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:19.145313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:19.262838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788029-7ecf-4581-9e7b-d86803f7ec88 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:19.297990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:19.303983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:19.304587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:19.304863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:19.305835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:19.306511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:19.306813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:19.308199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:19.317379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 567/1150] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:17:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:19.324592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:19.372234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:19.427162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f2b9-a2bb-4d4b-a81d-407ccd937c09 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:19.465019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:19.465653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:19.465932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:19.466204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] PATH_INFO: `/v3/users/0f7dd171aa464b139d0a3a0c4b441e46` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:19.467370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] RBAC: Authorizing `identity:get_user(user_id=0f7dd171aa464b139d0a3a0c4b441e46)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:19.469071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:19.470561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 584/1151] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:19 2022] GET /identity/v3/users/0f7dd171aa464b139d0a3a0c4b441e46 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:19.477869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:19.599436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1e10-965c-43f5-9e15-259bff45a4bb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:19.638714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f986-5f61-4c24-9369-e8fbdd07646a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:19.639373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f986-5f61-4c24-9369-e8fbdd07646a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:19.640171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f986-5f61-4c24-9369-e8fbdd07646a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:19.640421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f986-5f61-4c24-9369-e8fbdd07646a None admin] PATH_INFO: `/v3/projects/53a578f745cc4051a984cf10bbc4c264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:19.641633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f986-5f61-4c24-9369-e8fbdd07646a None admin] RBAC: Authorizing `identity:get_project(project_id=53a578f745cc4051a984cf10bbc4c264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:19.643055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f986-5f61-4c24-9369-e8fbdd07646a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:19.644686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 568/1152] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:19 2022] GET /identity/v3/projects/53a578f745cc4051a984cf10bbc4c264 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:19.650814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:19.766200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a028673-0832-4b24-babf-04426f9dc72c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:19.781320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:19.806691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3288d9-fa3d-4f0b-889e-09c6998f3938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:19.807352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3288d9-fa3d-4f0b-889e-09c6998f3938 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:19.807632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3288d9-fa3d-4f0b-889e-09c6998f3938 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:19.807906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3288d9-fa3d-4f0b-889e-09c6998f3938 None admin] PATH_INFO: `/v3/projects/53a578f745cc4051a984cf10bbc4c264/users/0f7dd171aa464b139d0a3a0c4b441e46/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:19.809919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3288d9-fa3d-4f0b-889e-09c6998f3938 None admin] RBAC: Authorizing `identity:create_grant(project_id=53a578f745cc4051a984cf10bbc4c264, user_id=0f7dd171aa464b139d0a3a0c4b441e46, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:19.811351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3288d9-fa3d-4f0b-889e-09c6998f3938 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:19.825240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 585/1153] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:17:19 2022] PUT /identity/v3/projects/53a578f745cc4051a984cf10bbc4c264/users/0f7dd171aa464b139d0a3a0c4b441e46/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:17:21.986200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338de1c-de1f-4839-9cb1-2c199e874e92 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:21.986763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338de1c-de1f-4839-9cb1-2c199e874e92 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:21.986763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338de1c-de1f-4839-9cb1-2c199e874e92 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:21.988084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 569/1154] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:21.993569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:21.994021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:21.994110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:22.012142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:22.013370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:22.082334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 586/1155] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:22.355042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338de1c-de1f-4839-9cb1-2c199e874e92 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:22.414905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:22.456639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338de1c-de1f-4839-9cb1-2c199e874e92 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:22.490187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:22.490803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:22.491080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:22.491347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:22.501676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:17:22.506429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:22.507539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:17:22.508584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 570/1156] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:17:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:22.513954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:22.610892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4b9bd-4fe0-4023-a2e6-a16fba35f91b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:22.613056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:22.644392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:22.644979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:22.645275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:22.645539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:22.646186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:22.646479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:22.647810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:22.656214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 587/1157] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:22.662234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:22.768541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b17b0c-708a-4ad5-b57d-064e1cb2f705 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:22.788372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:22.802404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:22.803025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:22.803301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:22.803578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] PATH_INFO: `/v3/users/0f7dd171aa464b139d0a3a0c4b441e46` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:22.804691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] RBAC: Authorizing `identity:get_user(user_id=0f7dd171aa464b139d0a3a0c4b441e46)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:22.806158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:22.807736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 571/1158] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:22 2022] GET /identity/v3/users/0f7dd171aa464b139d0a3a0c4b441e46 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:17:22.813594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:22.913061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c80b98-ca60-4bb4-b624-9f21c3255540 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:22.946779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:22.947372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:22.947649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:22.947919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:22.980758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:23.013495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:23.015446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:23.109403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 588/1159] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:17:23.115349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:23.214719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81103e52-f2f4-4cf5-adce-32bb5dcddb97 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:23.247170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6e610-7946-4502-a50d-8446e4e042a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:23.247788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6e610-7946-4502-a50d-8446e4e042a9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:23.248070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6e610-7946-4502-a50d-8446e4e042a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:23.248340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6e610-7946-4502-a50d-8446e4e042a9 None admin] PATH_INFO: `/v3/projects/53a578f745cc4051a984cf10bbc4c264` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:23.249967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6e610-7946-4502-a50d-8446e4e042a9 None admin] RBAC: Authorizing `identity:get_project(project_id=53a578f745cc4051a984cf10bbc4c264)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:23.251397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6e610-7946-4502-a50d-8446e4e042a9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:23.253037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 572/1160] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:23 2022] GET /identity/v3/projects/53a578f745cc4051a984cf10bbc4c264 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:23.259179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:23.321932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:23.356273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b06ac-4df1-400b-a82d-165f6e5ee4be None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:23.389639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:23.390722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:23.390996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:23.391268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:23.391983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=53a578f745cc4051a984cf10bbc4c264, user.id=0f7dd171aa464b139d0a3a0c4b441e46) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:23.392285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:23.393718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25435a6-2269-4659-938e-800958a1bd2d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:23.399316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 589/1161] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:17:23 2022] GET /identity/v3/role_assignments?user.id=0f7dd171aa464b139d0a3a0c4b441e46&scope.project.id=53a578f745cc4051a984cf10bbc4c264&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:25.580825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098fa94-e2b1-406f-b203-5ba96c02a5e6 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:25.581437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098fa94-e2b1-406f-b203-5ba96c02a5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:25.581437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098fa94-e2b1-406f-b203-5ba96c02a5e6 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:25.582748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 573/1162] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:25.588187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:25.588644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:25.588748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:25.607519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:25.609349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:25.676720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 590/1163] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:25.947625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098fa94-e2b1-406f-b203-5ba96c02a5e6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:26.017749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:26.065713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098fa94-e2b1-406f-b203-5ba96c02a5e6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:26.100309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:26.100914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:26.101222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:26.101497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:26.175988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:26.177490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:26.236210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:26.281954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 574/1164] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:17:26.287960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:26.350576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:26.392322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8959b7-899f-496e-813f-aca821e46245 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:26.424909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:26.425561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:26.425842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:26.426111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:26.432609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:26.434066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1c970642-5b08-4bed-81d0-e0855b160d8f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:26.446085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:26.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:17:26.448660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 591/1165] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:17:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:26.453886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:26.551885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fe728-709d-4dc7-8c3e-29ae6937a610 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:26.585485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:26.586151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:26.586374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:26.586641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:26.587283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:26.587580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:26.588932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0011-c482-4597-afcf-ec5634295009 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:26.596441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 575/1166] 10.222.0.85 () {56 vars in 1115 bytes} [Wed Nov 2 12:17:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:28.724918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4100b-87ef-454a-a58e-8f9c1c4c745d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:28.725476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4100b-87ef-454a-a58e-8f9c1c4c745d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:28.725476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4100b-87ef-454a-a58e-8f9c1c4c745d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:28.726796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 592/1167] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:28.732249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d47a3-98f5-454c-9d1d-7408f5673eab None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:28.732752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d47a3-98f5-454c-9d1d-7408f5673eab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:28.732853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d47a3-98f5-454c-9d1d-7408f5673eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:28.751691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-400d47a3-98f5-454c-9d1d-7408f5673eab None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:28.752866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-400d47a3-98f5-454c-9d1d-7408f5673eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:28.767266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:28.820120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 576/1168] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:29.089819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4100b-87ef-454a-a58e-8f9c1c4c745d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:29.188910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4100b-87ef-454a-a58e-8f9c1c4c745d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:29.209936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:29.225574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a7302-b9e3-4467-9c53-a9a518d3f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:29.226202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7302-b9e3-4467-9c53-a9a518d3f43b None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:29.226481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7302-b9e3-4467-9c53-a9a518d3f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:29.226755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7302-b9e3-4467-9c53-a9a518d3f43b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:29.227455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a7302-b9e3-4467-9c53-a9a518d3f43b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:29.228849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a7302-b9e3-4467-9c53-a9a518d3f43b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:29.246679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 593/1169] 10.222.0.85 () {60 vars in 1138 bytes} [Wed Nov 2 12:17:29 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:17:31.435366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0f001-1faa-40cd-b119-2fb8844ea5d2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:31.435920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0f001-1faa-40cd-b119-2fb8844ea5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:31.435920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0f001-1faa-40cd-b119-2fb8844ea5d2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:31.437321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 577/1170] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:31.442483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:31.442898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:31.443013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:31.460674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:31.461884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:31.526554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 594/1171] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:31.793987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0f001-1faa-40cd-b119-2fb8844ea5d2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:31.902654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0f001-1faa-40cd-b119-2fb8844ea5d2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:31.938060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9b1c4-b81d-4326-bebb-15151feedc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:31.938687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b1c4-b81d-4326-bebb-15151feedc81 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:31.938961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b1c4-b81d-4326-bebb-15151feedc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:31.939232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b1c4-b81d-4326-bebb-15151feedc81 None admin] PATH_INFO: `/v3/domains/df873dc11588484ab289b79ed7e4cbee` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:31.949806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9b1c4-b81d-4326-bebb-15151feedc81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=df873dc11588484ab289b79ed7e4cbee)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:31.951380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9b1c4-b81d-4326-bebb-15151feedc81 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:31.953549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 578/1172] 10.222.0.85 () {56 vars in 1183 bytes} [Wed Nov 2 12:17:31 2022] GET /identity/v3/domains/df873dc11588484ab289b79ed7e4cbee => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:31.959699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:32.062004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298fb6-2782-4d31-8804-f07f9df4acb1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:32.074213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:32.106809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12968a41-1e5e-4b24-b3a0-06e45e0522ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:32.107413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968a41-1e5e-4b24-b3a0-06e45e0522ce None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:32.107690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968a41-1e5e-4b24-b3a0-06e45e0522ce None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:32.107959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12968a41-1e5e-4b24-b3a0-06e45e0522ce None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:32.108659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12968a41-1e5e-4b24-b3a0-06e45e0522ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:32.110113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12968a41-1e5e-4b24-b3a0-06e45e0522ce None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:32.137429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 595/1173] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:17:31 2022] POST /identity/v3/projects => generated 361 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:17:34.379319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1530f-d710-4a0e-90d9-461985ef89a1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:34.379888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1530f-d710-4a0e-90d9-461985ef89a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:34.379888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1530f-d710-4a0e-90d9-461985ef89a1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:34.381234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 579/1174] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:34.387338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:34.387861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:34.387970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:34.407491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:34.408690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:34.478429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 596/1175] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:34.750761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1530f-d710-4a0e-90d9-461985ef89a1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:34.766198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:34.867914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1530f-d710-4a0e-90d9-461985ef89a1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:34.902797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55426-a5fb-4cf7-994a-37c01d1ad6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:34.903478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55426-a5fb-4cf7-994a-37c01d1ad6ca None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:34.903794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55426-a5fb-4cf7-994a-37c01d1ad6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:34.904105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55426-a5fb-4cf7-994a-37c01d1ad6ca None admin] PATH_INFO: `/v3/domains/df873dc11588484ab289b79ed7e4cbee` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:34.905879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55426-a5fb-4cf7-994a-37c01d1ad6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=df873dc11588484ab289b79ed7e4cbee)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:34.907356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55426-a5fb-4cf7-994a-37c01d1ad6ca None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:34.909265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 580/1176] 10.222.0.85 () {56 vars in 1183 bytes} [Wed Nov 2 12:17:34 2022] GET /identity/v3/domains/df873dc11588484ab289b79ed7e4cbee => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:17:34.917836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:35.016699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:35.026494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cc885-1608-48be-ad23-34b34daa9478 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:35.062145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886fb88-730c-433d-a571-ed6a897f7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:35.062754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886fb88-730c-433d-a571-ed6a897f7622 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:35.063027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886fb88-730c-433d-a571-ed6a897f7622 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:35.063291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886fb88-730c-433d-a571-ed6a897f7622 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:35.063952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886fb88-730c-433d-a571-ed6a897f7622 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:35.065406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886fb88-730c-433d-a571-ed6a897f7622 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:35.087037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 597/1177] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:17:34 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:17:37.418230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8666a-61af-435a-8d44-fa02fbcc6dd5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:37.418802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8666a-61af-435a-8d44-fa02fbcc6dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:37.418802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8666a-61af-435a-8d44-fa02fbcc6dd5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:37.420117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 581/1178] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:37.426201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:37.426752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:37.426752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:37.445918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:37.447250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:37.526257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 598/1179] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:17:37.816463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8666a-61af-435a-8d44-fa02fbcc6dd5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:37.904971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:37.930899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8666a-61af-435a-8d44-fa02fbcc6dd5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:37.968293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:37.968924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:37.969263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:37.969535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:37.981086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:17:37.986583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:37.988503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:17:37.990522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 582/1180] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:37.996211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:38.084597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:38.110497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4cead-dd96-4725-8650-526c00b84510 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:38.149679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:38.150871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:38.151355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:38.151914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:38.153273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:38.154010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:38.157402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:38.180819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 599/1181] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:38.188526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:38.309293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a9fd-a5b9-47af-81a2-8daa4e9e6d35 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:38.360892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:38.361658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:38.362128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:38.362579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] PATH_INFO: `/v3/users/8c7d4b28e5f34530bf64607c5ba9fe56` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:38.375805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:38.385720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] RBAC: Authorizing `identity:get_user(user_id=8c7d4b28e5f34530bf64607c5ba9fe56)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:38.387842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:38.390845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 583/1182] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:38 2022] GET /identity/v3/users/8c7d4b28e5f34530bf64607c5ba9fe56 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:38.399056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:38.551583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050cb22-13df-4c17-8b5b-a64c65b7996d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:38.596950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:38.597652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:38.597931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:38.598217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:38.611390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:38.679241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:38.680975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:38.813386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 600/1183] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:17:38.823011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:38.973976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d23be-fc87-44ba-8905-4dce74f9cc31 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:39.014076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257b7b5-dcdb-4d7b-96e0-2d29e0bb6998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:39.015335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b7b5-dcdb-4d7b-96e0-2d29e0bb6998 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:39.015712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b7b5-dcdb-4d7b-96e0-2d29e0bb6998 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:39.016050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b7b5-dcdb-4d7b-96e0-2d29e0bb6998 None admin] PATH_INFO: `/v3/projects/fffac95341ab4320b8115d803b152011` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:39.017398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257b7b5-dcdb-4d7b-96e0-2d29e0bb6998 None admin] RBAC: Authorizing `identity:get_project(project_id=fffac95341ab4320b8115d803b152011)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:39.019056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257b7b5-dcdb-4d7b-96e0-2d29e0bb6998 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:39.020866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 584/1184] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:38 2022] GET /identity/v3/projects/fffac95341ab4320b8115d803b152011 => generated 361 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:39.027958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:39.086315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:39.195985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6cbfd-3319-44f2-991e-c76588af6aa3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:39.244625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:39.245280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:39.245600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:39.245869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:39.246674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=fffac95341ab4320b8115d803b152011, user.id=8c7d4b28e5f34530bf64607c5ba9fe56) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:39.247012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:39.248546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d1541-d80b-4369-86ee-92e60e40441e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:39.255258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 601/1185] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:17:39 2022] GET /identity/v3/role_assignments?user.id=8c7d4b28e5f34530bf64607c5ba9fe56&scope.project.id=fffac95341ab4320b8115d803b152011&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:17:41.529168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a00c0-1ee4-4cd5-92a9-4edb35d9c073 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:41.529689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a00c0-1ee4-4cd5-92a9-4edb35d9c073 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:41.529689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a00c0-1ee4-4cd5-92a9-4edb35d9c073 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:41.531245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 585/1186] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:41.538506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:41.539087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:41.539087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:41.559859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:41.561821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:41.641290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 602/1187] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:41.929361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a00c0-1ee4-4cd5-92a9-4edb35d9c073 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:41.999128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:42.044341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a00c0-1ee4-4cd5-92a9-4edb35d9c073 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:42.087238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:42.088107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:42.088454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:42.088769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:42.101981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:17:42.107712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:42.108920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:17:42.110011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 586/1188] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:42.115097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:42.154252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:42.230711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a4a6-011d-4994-875c-43f4f1ec55e1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:42.268978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:42.269617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:42.269891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:42.270161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:42.270832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:42.271177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:42.272589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:42.282341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 603/1189] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:17:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:42.288595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:42.406667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208c1fc-c3d7-4be5-b89f-91f3ea736a56 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:42.432064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:42.444551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:42.445211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:42.445490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:42.445761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] PATH_INFO: `/v3/users/8c7d4b28e5f34530bf64607c5ba9fe56` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:42.446902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] RBAC: Authorizing `identity:get_user(user_id=8c7d4b28e5f34530bf64607c5ba9fe56)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:42.448761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:42.450624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 587/1190] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:42 2022] GET /identity/v3/users/8c7d4b28e5f34530bf64607c5ba9fe56 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:42.458351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:42.569167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:42.575830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1598f-dd7a-4c11-87be-1822ac11ab43 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:42.619334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96960b-8d7c-4e76-b0f2-7a0daced4971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:42.620058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96960b-8d7c-4e76-b0f2-7a0daced4971 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:42.620392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96960b-8d7c-4e76-b0f2-7a0daced4971 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:42.620724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96960b-8d7c-4e76-b0f2-7a0daced4971 None admin] PATH_INFO: `/v3/projects/fffac95341ab4320b8115d803b152011` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:42.622511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96960b-8d7c-4e76-b0f2-7a0daced4971 None admin] RBAC: Authorizing `identity:get_project(project_id=fffac95341ab4320b8115d803b152011)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:42.625782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96960b-8d7c-4e76-b0f2-7a0daced4971 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:42.628680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 604/1191] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:42 2022] GET /identity/v3/projects/fffac95341ab4320b8115d803b152011 => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:42.635160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:42.745763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7856f0-85b9-4b89-a8a2-3ad9f096772f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:42.781136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b29a42-02f1-457e-8740-c412fd85bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:42.781784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b29a42-02f1-457e-8740-c412fd85bd0f None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:42.782059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b29a42-02f1-457e-8740-c412fd85bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:42.782329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b29a42-02f1-457e-8740-c412fd85bd0f None admin] PATH_INFO: `/v3/projects/fffac95341ab4320b8115d803b152011/users/8c7d4b28e5f34530bf64607c5ba9fe56/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:42.784277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b29a42-02f1-457e-8740-c412fd85bd0f None admin] RBAC: Authorizing `identity:create_grant(project_id=fffac95341ab4320b8115d803b152011, user_id=8c7d4b28e5f34530bf64607c5ba9fe56, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:42.785852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b29a42-02f1-457e-8740-c412fd85bd0f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:42.799739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 588/1192] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:17:42 2022] PUT /identity/v3/projects/fffac95341ab4320b8115d803b152011/users/8c7d4b28e5f34530bf64607c5ba9fe56/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:17:45.112184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e7299-acf4-4cfa-8523-6d35674500b3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:45.112755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e7299-acf4-4cfa-8523-6d35674500b3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:45.112880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e7299-acf4-4cfa-8523-6d35674500b3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:45.114410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 605/1193] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:17:45.120583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b345-9323-4818-8741-0c32eedd623e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:45.121074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b345-9323-4818-8741-0c32eedd623e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:45.121168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b345-9323-4818-8741-0c32eedd623e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:45.142072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4593b345-9323-4818-8741-0c32eedd623e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:17:45.143283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4593b345-9323-4818-8741-0c32eedd623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:17:45.201430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:45.226867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 589/1194] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:17:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:17:45.527902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e7299-acf4-4cfa-8523-6d35674500b3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:45.643080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e7299-acf4-4cfa-8523-6d35674500b3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:45.677515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:45.678138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:45.678414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:45.678684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:45.689486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:17:45.695716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:17:45.697741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:17:45.699555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 606/1195] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:17:45.704683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b345-9323-4818-8741-0c32eedd623e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:45.813760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b345-9323-4818-8741-0c32eedd623e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:45.851959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:45.852566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:45.852844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:45.853147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:45.853821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:45.854122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:45.855489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:45.865007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 590/1196] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:45.871662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:45.901737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:46.017160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7174b-d07c-4aed-989b-af4c759355dc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:46.055823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:46.056453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:46.056755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:46.057077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] PATH_INFO: `/v3/users/8c7d4b28e5f34530bf64607c5ba9fe56` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:46.058320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] RBAC: Authorizing `identity:get_user(user_id=8c7d4b28e5f34530bf64607c5ba9fe56)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:46.059788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:46.062227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 607/1197] 10.222.0.85 () {56 vars in 1177 bytes} [Wed Nov 2 12:17:45 2022] GET /identity/v3/users/8c7d4b28e5f34530bf64607c5ba9fe56 => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:46.068370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:46.118008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:46.179910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e26b1-f7f1-4d1f-a583-4912d393d215 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:46.216794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:46.217531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:46.217531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:46.217531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:46.300751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:46.302194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:46.349592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:46.415262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 591/1198] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:17:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 12:17:46.423836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:46.525195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:46.535849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a73b44-92b1-4f16-9529-ec97712e9700 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:46.574072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dec96-9b84-4755-9f77-ea0f3adf72fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:46.574704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dec96-9b84-4755-9f77-ea0f3adf72fe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:46.575005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dec96-9b84-4755-9f77-ea0f3adf72fe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:46.575279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dec96-9b84-4755-9f77-ea0f3adf72fe None admin] PATH_INFO: `/v3/projects/fffac95341ab4320b8115d803b152011` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:46.576425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dec96-9b84-4755-9f77-ea0f3adf72fe None admin] RBAC: Authorizing `identity:get_project(project_id=fffac95341ab4320b8115d803b152011)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:46.577891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dec96-9b84-4755-9f77-ea0f3adf72fe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:46.579557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 608/1199] 10.222.0.85 () {56 vars in 1186 bytes} [Wed Nov 2 12:17:46 2022] GET /identity/v3/projects/fffac95341ab4320b8115d803b152011 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:46.586103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:17:46.696845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761dee2-316c-4e02-ad2e-5ee65eabcd7c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:17:46.734578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:17:46.735220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:17:46.735527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:17:46.735838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:17:46.736557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=fffac95341ab4320b8115d803b152011, user.id=8c7d4b28e5f34530bf64607c5ba9fe56) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:17:46.736895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:17:46.738393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7f442-c39a-4970-88ec-d0cf2716c118 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:17:46.744710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 592/1200] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:17:46 2022] GET /identity/v3/role_assignments?user.id=8c7d4b28e5f34530bf64607c5ba9fe56&scope.project.id=fffac95341ab4320b8115d803b152011&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:17:56.568907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=20 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:17:56.741928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:45.193624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3ee9-687d-402d-bf71-885b94afc95c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:45.194284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3ee9-687d-402d-bf71-885b94afc95c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:45.194473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3ee9-687d-402d-bf71-885b94afc95c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:45.196289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 609/1201] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:18:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:18:45.203780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:45.204312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:45.204399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:45.226868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:18:45.228304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:18:45.338082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 593/1202] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:18:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:18:45.674146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3ee9-687d-402d-bf71-885b94afc95c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:45.867230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3ee9-687d-402d-bf71-885b94afc95c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:45.943292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:45.943292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:45.943292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:45.943292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:45.962615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:45.966258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:18:45.998065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:46.000245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:18:46.002588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:18:46.002588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:46.002588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:18:46.002588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 610/1203] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:18:46.008114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:46.183388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56034b0-bd3c-4bf4-9105-c8f800a02442 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:46.231516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:46.241872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:46.242548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:46.242839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:46.243069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:46.243719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:46.244017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:46.245492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a292ee-7d3c-41cd-bc69-c8666598e9da None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:46.261051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 594/1204] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:46.266072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:46.349727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:46.409041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1e276-19df-4a5e-a92c-5f723c42c6ec None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:46.452390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954e2f2-c455-40af-ba10-c3046b934e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:46.453121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954e2f2-c455-40af-ba10-c3046b934e69 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:46.453887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954e2f2-c455-40af-ba10-c3046b934e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:46.453887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954e2f2-c455-40af-ba10-c3046b934e69 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:46.454366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954e2f2-c455-40af-ba10-c3046b934e69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:46.455998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954e2f2-c455-40af-ba10-c3046b934e69 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:46.490382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 611/1205] 10.222.0.85 () {60 vars in 1132 bytes} [Wed Nov 2 12:18:46 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:18:49.209322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2f417-633a-45af-96d5-e459784ec440 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:49.209884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2f417-633a-45af-96d5-e459784ec440 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:49.210047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2f417-633a-45af-96d5-e459784ec440 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:49.211971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 595/1206] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:18:49.218534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:49.219468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:49.219468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:49.241320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:18:49.242544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:18:49.331866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 612/1207] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:18:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:18:49.636921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2f417-633a-45af-96d5-e459784ec440 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:49.812859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2f417-633a-45af-96d5-e459784ec440 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:49.864953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:49.866695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:49.867526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:49.868256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:49.888485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:18:49.896221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:49.898171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:49.899714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 596/1208] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:18:49.909154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:49.961530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:50.052610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8483a58-7f71-4dda-8f67-a5a519ceb600 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:50.102116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:50.103267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:50.103267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:50.103267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:50.103807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:50.104104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:50.105805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:50.115421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 613/1209] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:18:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:50.121598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:50.191253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:50.270592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec716b-723d-40ec-bd5d-c766698f84cc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:50.320728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:50.321512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:50.321833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:50.322104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:50.335146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:50.336645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:18:50.354028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:50.355093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:18:50.357401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:18:50.357401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:50.357401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:50.360420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 597/1210] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:18:50.362040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:50.495281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:50.499187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee01a-6510-4001-8c67-b7a39bcc0b2c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:50.545945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:50.546567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:50.546898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:50.547172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:50.547957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:50.548344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:50.550357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:50.565241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 614/1211] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:50.571872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:50.753322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b849f-0c76-4179-9d31-4370469d4ae8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:50.755015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:50.806922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:50.807952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:50.808504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:50.809631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:50.823886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:50.826398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:18:50.852742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:50.854069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:50.860400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 598/1212] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:18:50.863056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:51.005593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6438-4bd0-42bd-9c55-22fa1c51cf3d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:51.062602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:51.063490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:51.064190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:51.064331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:51.065614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:51.066005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:51.067629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:51.099180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 615/1213] 10.222.0.85 () {56 vars in 1143 bytes} [Wed Nov 2 12:18:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:51.108925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:51.238857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1da82-e949-4a91-b411-a1d95486e4d9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:51.286694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:51.287588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:51.287793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:51.288043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:51.321429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:51.370492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:51.372421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:51.477777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 599/1214] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:18:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:18:51.483792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:51.505283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:51.605181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d5d53-cdd5-43a7-bf4d-974ea24c03a8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:51.648651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:51.649321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:51.649592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:51.649865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:51.660841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:51.662328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:18:51.680323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:51.681887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:18:51.683004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:18:51.683004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:51.683004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:18:51.683004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 616/1215] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:18:51.688032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:51.761642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:51.811750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfa85d-0588-4dac-b8fd-f5ee4daa7c81 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:51.851833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:51.852417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:51.852832 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:51.853036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:51.853658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:51.853965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:51.855487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:51.868698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 600/1216] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:51.875140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:51.969825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:52.010575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efbe10-df72-45b7-b7ca-a790fedb7c52 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:52.056555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:52.057327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:52.057701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:52.058154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:52.069595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:52.071141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:18:52.083889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:52.085060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:18:52.086159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 617/1217] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:18:52.091600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:52.211643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d1edc-f9c9-47db-8f89-d651b763dcae None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:52.237997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:52.254820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:52.255715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:52.256019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:52.256457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:52.257581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:52.257911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:52.259373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5be5f5-f6c9-4495-86f7-0f63b03b859e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:52.275969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 601/1218] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:52.283529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:52.440128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6647a-d6f5-42ee-8c9d-bfc9873c4b48 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:52.472789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:52.481707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:52.482337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:52.482595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:52.482863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:52.484210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=1de535a7ce1e475abdb02308cb3326e7) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:52.484210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:52.485484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9769eb-2350-400f-8032-a3deb151f7e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:52.492511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 618/1219] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:18:52 2022] GET /identity/v3/role_assignments?user.id=1de535a7ce1e475abdb02308cb3326e7&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:54.908968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f43f0-6fc8-4ff6-9594-7e4a982ebc40 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:54.909757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f43f0-6fc8-4ff6-9594-7e4a982ebc40 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:54.909757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f43f0-6fc8-4ff6-9594-7e4a982ebc40 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:54.911475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 602/1220] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:18:54.919958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:54.920609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:54.920609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:54.941581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:18:54.942968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:18:55.023704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 619/1221] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:18:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:18:55.312344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f43f0-6fc8-4ff6-9594-7e4a982ebc40 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:55.426494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f43f0-6fc8-4ff6-9594-7e4a982ebc40 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:55.467693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:55.468367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:55.468643 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:55.468915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:55.480815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:18:55.486393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:55.487698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:55.489052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 603/1222] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:18:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:18:55.494363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:55.624118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c835f-514d-4d6a-8620-8d01c1871d39 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:55.652242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:55.664962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:55.665620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:55.665881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:55.666153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:55.666818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:55.667118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:55.669464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:55.678592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 620/1223] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:18:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:55.685957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:55.743517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:55.813257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fbfa2-74d8-492f-a444-cb396d5273a5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:55.850900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:55.851599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:55.851896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:55.852359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:55.865283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:55.867108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:18:55.884670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:55.886245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:18:55.887191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:18:55.887191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:55.887191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:55.887191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 604/1224] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:18:55.892343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:56.013102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d866d-9b23-4ec5-9139-14d2e9c0b683 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:56.051510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:56.052177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:56.052452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:56.052722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:56.053421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:56.053774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:56.055262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:56.069675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 621/1225] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:56.075863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:56.184240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:56.192300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df295-e83b-4537-bf72-23b4062081ff None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:56.230023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:56.230722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:56.230992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:56.231254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:56.239900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:56.241406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:18:56.253629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:56.254857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:18:56.256060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:56.257299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 605/1226] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:18:56.261021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:56.313793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:56.371741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28e3ab-ca49-4ccc-addd-428623c8725c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:56.407363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:56.407966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:56.408239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:56.408506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:56.409262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:56.409560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:56.410966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:56.429924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 622/1227] 10.222.0.85 () {56 vars in 1143 bytes} [Wed Nov 2 12:18:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:56.437090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:56.555487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a4ff-0379-4ed2-a19a-9c6b3c6446d1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:56.594807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:56.595491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:56.595761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:56.596022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:56.607526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:56.608965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:18:56.624632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:56.625924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:18:56.627175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:18:56.627175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:18:56.627175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:56.627175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 606/1228] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:18:56.632285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:56.747309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0c1d2-f89a-4947-b989-516d6fb993d2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:56.749858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:56.784571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:56.785213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:56.785490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:56.785764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:56.786405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:56.786750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:56.788218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:56.801429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 623/1229] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:56.807439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:56.817962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:56.924954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc8299-cac5-4627-b207-848abc09b2dd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:56.968465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:56.969400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:56.969781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:56.970140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:56.978298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:56.979769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:18:56.991987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:18:56.993218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:18:56.994238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 607/1230] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:18:56.999606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:57.127982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81474aa8-0ce8-417c-b214-f9494d7f6ffd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:57.170498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:57.171238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:57.171552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:57.171874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:57.172716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:18:57.173081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:57.174967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9589980-9aca-4bab-b292-ca59a2885190 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:57.191286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 624/1231] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:18:57.198044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:18:57.273029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:18:57.311901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d357-d580-4e4c-9938-91967e9b65e8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:18:57.350950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357fdd8-8bba-4468-a7b9-c78f44e4231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:18:57.351692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357fdd8-8bba-4468-a7b9-c78f44e4231b None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:57.351970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357fdd8-8bba-4468-a7b9-c78f44e4231b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:57.352244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357fdd8-8bba-4468-a7b9-c78f44e4231b None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/1de535a7ce1e475abdb02308cb3326e7/roles/02a292c24f304d06a8ec62f1df807dfb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:57.368147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357fdd8-8bba-4468-a7b9-c78f44e4231b None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=1de535a7ce1e475abdb02308cb3326e7, role_id=02a292c24f304d06a8ec62f1df807dfb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:18:57.369662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357fdd8-8bba-4468-a7b9-c78f44e4231b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:18:57.382492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 608/1232] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:18:57 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/1de535a7ce1e475abdb02308cb3326e7/roles/02a292c24f304d06a8ec62f1df807dfb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:18:59.658785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd92d9-05ef-4fa4-927f-031477751b2d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:59.659338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd92d9-05ef-4fa4-927f-031477751b2d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:59.659442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd92d9-05ef-4fa4-927f-031477751b2d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:59.660927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 625/1233] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:18:59.667297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:18:59.667747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:18:59.667971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:18:59.687604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:18:59.688903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:18:59.772067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 609/1234] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:18:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:00.065623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd92d9-05ef-4fa4-927f-031477751b2d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:00.093993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:00.180195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd92d9-05ef-4fa4-927f-031477751b2d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:00.216799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:00.217454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:00.217740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:00.218124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:00.229030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:19:00.234365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:00.235471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:00.236354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 626/1235] 10.222.0.85 () {56 vars in 1102 bytes} [Wed Nov 2 12:19:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:19:00.241871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:00.300230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:00.369833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8fd2a-5328-4d67-82c3-6fb148e8832a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:00.419625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:00.420241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:00.420511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:00.420774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:00.421483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:00.421776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:00.423264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:00.433542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 610/1236] 10.222.0.85 () {56 vars in 1103 bytes} [Wed Nov 2 12:19:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:00.442019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:00.544831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:00.564151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cbb00-3fd9-4f0e-a451-b9cf62770b7b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:00.605060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:00.605743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:00.606021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:00.606295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:00.618182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:00.619818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:00.637691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:00.639273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:00.640526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:00.640526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:00.640526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:00.640526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 627/1237] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:00.644577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:00.769634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e39c1-d40d-4e32-b333-4ce0459dcbcc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:00.780379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:00.810362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:00.811005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:00.811345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:00.811646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:00.812415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:00.812748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:00.814372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:00.833917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 611/1238] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:00.835616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:00.961607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65a984-6f6a-4348-ad70-0f31754b2e6f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:01.004295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:01.004929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:01.005257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:01.005509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:01.008633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:01.014799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:01.016435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:19:01.029063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:01.030410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:01.031791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:01.033303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 628/1239] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:01.036088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:01.149615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a8b76-d3d5-4cb4-ba12-5598c0016556 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:01.187415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:01.188019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:01.188284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:01.188548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:01.189307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:01.189604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:01.191025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:01.211170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 612/1240] 10.222.0.85 () {56 vars in 1143 bytes} [Wed Nov 2 12:19:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:01.217667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:01.336948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d337841-f0fe-4c2c-b672-f5a381863f17 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:01.376279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:01.376940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:01.377262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:01.377523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:01.448741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:01.458003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:01.459613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:01.574101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 629/1241] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:19:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:19:01.580190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:01.615611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:01.697178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc4608-1d29-4898-8b64-4e00674b4213 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:01.738920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:01.739403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:01.739680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:01.739954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:01.753237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:01.754865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:01.773721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:01.780940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:01.781942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:01.781942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:01.781942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:01.781942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 613/1242] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:01.782206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:01.886419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:01.894048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e6da8-7841-44ec-ad44-70724f37ce0e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:01.933817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:01.934551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:01.934862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:01.935158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:01.935908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:01.936236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:01.937710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:01.952003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 630/1243] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:01.959918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:02.050595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:02.075452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3ee12-557d-4ffe-a325-47d734830224 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:02.114224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:02.114850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:02.115123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:02.115419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:02.122985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:02.124423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:19:02.137863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:02.138946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:02.139977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 614/1244] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:19:02.150116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:02.271474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553be3-21b8-4b74-b30e-b70a9cc16185 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:02.312576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:02.313373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:02.313779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:02.314249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:02.315219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:02.315652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:02.317641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9e914-6fd9-4340-bbb4-6e14c2e03aa8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:02.331968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 631/1245] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:02.338416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:02.458069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc40-a1b4-48a8-a1b6-b2cef660ea92 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:02.503218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:02.503889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:02.504304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:02.504584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:02.505414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] RBAC: Adding query filter params (role.id=02a292c24f304d06a8ec62f1df807dfb, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=1de535a7ce1e475abdb02308cb3326e7) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:02.505725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:02.507301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3033-c5cf-48d7-b203-9dfddce3f745 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:02.514937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 615/1246] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:19:02 2022] GET /identity/v3/role_assignments?user.id=1de535a7ce1e475abdb02308cb3326e7&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=02a292c24f304d06a8ec62f1df807dfb => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:04.804450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2695a5-01ae-4f4c-892a-396d824a58ae None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:04.805223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2695a5-01ae-4f4c-892a-396d824a58ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:04.805223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2695a5-01ae-4f4c-892a-396d824a58ae None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:04.806243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 632/1247] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:19:04.812610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:04.813128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:04.813274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:04.822322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:04.837185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:04.838470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:04.924963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 616/1248] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:05.217493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2695a5-01ae-4f4c-892a-396d824a58ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:05.238930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:05.352136 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2695a5-01ae-4f4c-892a-396d824a58ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:05.391547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:05.392243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:05.392621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:05.392918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:05.404317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:19:05.409989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:05.411244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:05.412312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 633/1249] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:05.417717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:05.551802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cdeed-017b-4370-873b-7f8c2ace86e7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:05.590646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:05.597920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:05.598741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:05.599110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:05.599525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:05.600333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:05.600637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:05.602439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:05.614157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 617/1250] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:05.622003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:05.729147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:05.757425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b037e-b8dd-4ceb-84df-04e549ec3119 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:05.798562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:05.799234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:05.799548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:05.799834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:05.811635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:05.813198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:05.829372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:05.830494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:05.831386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:05.831386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:05.831386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:05.831386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 634/1251] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:19:05.836454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:05.954662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfbf1f-c479-40d9-b1e8-7c0487eb093d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:05.991104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:05.991632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:05.991908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:05.992251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:05.992883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:05.993272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:05.994761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:06.007749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 618/1252] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:19:06.015522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:06.149888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73216-b178-425b-870e-92154d5fa8b7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:06.180606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:06.190241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:06.190907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:06.191203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:06.191474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:06.201424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:06.202978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:19:06.216232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:06.218219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:06.220198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 635/1253] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:06.223814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:06.300659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:06.354911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2995e-b08b-483b-bcc7-37da116044ad None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:06.395805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:06.397217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:06.397217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:06.397217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:06.398011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:06.399156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:06.400518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:06.425509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 619/1254] 10.222.0.85 () {56 vars in 1143 bytes} [Wed Nov 2 12:19:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:06.432008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:06.569146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df711258-53f8-40a6-b2df-81a5450b01a5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:06.619851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:06.620694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:06.621248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:06.621636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:06.700781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:06.716190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:06.717826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:06.840627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 636/1255] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:19:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:19:06.846543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:06.977366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3b59-84b0-450a-9dc4-527c50272b00 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:06.994115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:07.021186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:07.021873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:07.022183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:07.022546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:07.035108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:07.036878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:07.053794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:07.059475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:07.060447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:07.060447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:07.060447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:07.060447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 620/1256] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:19:07.061278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:07.170319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:07.184909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487ed29-c51c-48b5-b0a1-a542bd685648 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:07.224384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:07.225030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:07.225316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:07.225589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:07.226250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:07.226554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:07.227960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:07.241034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 637/1257] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:07.247094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:07.368354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4645b7-fcde-4ccb-bdcc-3faf3a787fe3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:07.405864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:07.406502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:07.406772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:07.407035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:07.414687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:07.416151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:19:07.428494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:07.430369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:07.432231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 621/1258] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:19:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:19:07.436683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:07.547108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0662-766e-451a-97a2-5a32f4f11b44 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:07.586577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:07.587195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:07.587470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:07.587736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:07.588419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:07.588722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:07.590203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80388f6f-ef25-4d70-8da5-c1fd3f7033e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:07.602314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 638/1259] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:19:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:07.608810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:07.637929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:07.736699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f7465-529a-4852-86fe-dde970ce693c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:07.785116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:07.785724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:07.786020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:07.786289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:07.787012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=1de535a7ce1e475abdb02308cb3326e7) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:07.787316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:07.788750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4066d9-7446-4537-b81c-a3dc3414ac85 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:07.799674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 622/1260] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:19:07 2022] GET /identity/v3/role_assignments?user.id=1de535a7ce1e475abdb02308cb3326e7&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:09.958220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98bc5-f01e-4c6a-9cb8-706ef41b78df None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:09.958749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98bc5-f01e-4c6a-9cb8-706ef41b78df None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:09.958749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98bc5-f01e-4c6a-9cb8-706ef41b78df None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:09.960076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 639/1261] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:19:09.965208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:09.965640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:09.965780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:09.984570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:09.985803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:10.066038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 623/1262] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:10.340909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af98bc5-f01e-4c6a-9cb8-706ef41b78df None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:10.351629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:10.441370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af98bc5-f01e-4c6a-9cb8-706ef41b78df None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:10.474482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:10.475109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:10.475412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:10.475679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:10.485819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:19:10.490601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:10.491701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:10.492734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 640/1263] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:19:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:10.498287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:10.517187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:10.596476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f9d4f-407a-4391-b701-0e1d6aa004ed None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:10.630188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:10.630775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:10.631047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:10.631311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:10.631961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:10.632257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:10.633634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:10.642340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 624/1264] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:10.648209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:10.715609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:10.756361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a44d7-74ba-4a8c-84d8-3169d72e9943 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:10.796889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:10.797547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:10.797825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:10.798092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:10.808088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:10.810152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:10.825243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:10.826444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:10.827376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:10.827376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:10.827376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:10.827376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 641/1265] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:10.832352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:10.911953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:10.931784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935a20f-ce77-4526-b61a-394fd8899e57 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:10.965642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:10.966252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:10.966493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:10.966759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:10.967386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:10.967681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:10.969057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:10.982962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 625/1266] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:10.988670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:11.087006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9f2a0-4801-4361-b062-887956a591f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:11.107042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:11.120958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:11.121636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:11.121921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:11.122374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:11.129621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:11.131032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:19:11.142215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:11.143225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:11.144280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 642/1267] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:11.149194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:11.248040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702f04e-4956-4ffa-bf55-c132a7954dd4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:11.285790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:11.286408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:11.286684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:11.286951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:11.287693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:11.287997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:11.289532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:11.300235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:11.309938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 626/1268] 10.222.0.85 () {56 vars in 1143 bytes} [Wed Nov 2 12:19:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:11.315970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:11.417265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943d021-5788-48b7-80ec-7c5d5008a53f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:11.451915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:11.452625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:11.452932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:11.453284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:11.463800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:11.465852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:11.480470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:11.481697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:11.482794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:11.482794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:11.482794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:11.482794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 643/1269] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:11.487461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:11.586274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45172-0331-4658-8f92-db29cae41ab6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:11.620008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:11.620608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:11.620883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:11.621183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:11.621822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:11.622123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:11.623481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:11.634455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 627/1270] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:11.640912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:11.672020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:11.744756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed45040-aef3-4dd5-9390-a78d93dd4086 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:11.780585 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:11.781238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:11.781518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:11.781826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:11.788417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:11.789885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:19:11.801048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:11.802124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:11.803208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 644/1271] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:11.808315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:11.867154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:11.909887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6db8f-767b-438d-a4a8-6643dbbaa0b0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:11.943963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:11.944563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:11.944840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:11.945141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:11.945830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:11.946140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:11.947570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d11eb8-c831-4295-a194-bfdb5d7c4118 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:11.960175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 628/1272] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:11.966400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:12.055780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:12.065973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8d11-2c15-4a5f-8896-2e232cde3db2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:12.098887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83a11a-4131-4fe8-887c-a824b6bebfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:12.099526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83a11a-4131-4fe8-887c-a824b6bebfc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:12.099800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83a11a-4131-4fe8-887c-a824b6bebfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:12.100067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83a11a-4131-4fe8-887c-a824b6bebfc1 None admin] PATH_INFO: `/v3/projects/0c95872158724e60be029b048d666cf7/users/1de535a7ce1e475abdb02308cb3326e7/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:12.101955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83a11a-4131-4fe8-887c-a824b6bebfc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c95872158724e60be029b048d666cf7, user_id=1de535a7ce1e475abdb02308cb3326e7, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:12.103357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83a11a-4131-4fe8-887c-a824b6bebfc1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:12.116108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 645/1273] 10.222.0.85 () {58 vars in 1439 bytes} [Wed Nov 2 12:19:11 2022] PUT /identity/v3/projects/0c95872158724e60be029b048d666cf7/users/1de535a7ce1e475abdb02308cb3326e7/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:19:14.292471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186c94-4f8e-48e5-8048-5668727b6252 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:14.293069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186c94-4f8e-48e5-8048-5668727b6252 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:14.293069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186c94-4f8e-48e5-8048-5668727b6252 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:14.294978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 629/1274] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:19:14.300581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:14.301110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:14.301110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:14.320352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:14.321590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:14.389016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 646/1275] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:14.658675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186c94-4f8e-48e5-8048-5668727b6252 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:14.783332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186c94-4f8e-48e5-8048-5668727b6252 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:14.813610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:14.832024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:14.832645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:14.832919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:14.833212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:14.845033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 12:19:14.849945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:14.852044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:14.853948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 630/1276] 10.222.0.85 () {56 vars in 1096 bytes} [Wed Nov 2 12:19:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:14.857429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:14.939278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:14.958640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ed6d2-d6f2-4636-a97b-052ffbfe9894 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:14.992498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:14.993122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:14.993400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:14.993677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:14.994331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:14.994630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:14.995979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:15.003804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 647/1277] 10.222.0.85 () {56 vars in 1099 bytes} [Wed Nov 2 12:19:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:15.009730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:15.109531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8295ac9-06bd-4e82-8e35-ce3a17145e1e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:15.143864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:15.144475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:15.144747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:15.145024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:15.155250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:15.156680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:15.172250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:15.174124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:15.175345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:15.175345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:15.175345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:15.175345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 631/1278] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:15.179463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:15.284146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e791fff-5aae-4aac-80aa-1d8f87399f0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:15.316639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:15.317330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:15.317508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:15.317780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:15.318403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:15.318698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:15.320025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:15.332062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 648/1279] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:15.337830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:15.368026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:15.444095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156735-4a07-4233-bdf8-cdacce190437 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:15.478207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:15.478833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:15.479162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:15.479438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:15.487728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:15.489238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 12:19:15.500688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:15.501840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:15.503195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 632/1280] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 12:19:15.507697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:15.511716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:15.615799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232673c5-f071-4249-97c5-a583da6bd794 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:15.655936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:15.656543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:15.656815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:15.657109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:15.657842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:15.658139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:15.659553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:15.678786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 649/1281] 10.222.0.85 () {56 vars in 1143 bytes} [Wed Nov 2 12:19:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:15.686547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:15.788261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:15.798781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71abe-70d0-4856-a415-ef2984b56d74 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:15.833018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:15.833612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:15.833887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:15.834155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:15.904181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:15.905582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:15.995166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:16.005714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 633/1282] 10.222.0.85 () {58 vars in 1282 bytes} [Wed Nov 2 12:19:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 12:19:16.012386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:16.074253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:16.113187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89c85-30e6-4edd-9b3f-a1fbe0bb5662 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:16.146204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:16.146812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:16.147087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:16.147352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:16.157796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:16.159157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 12:19:16.173913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:16.175061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 12:19:16.175981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 12:19:16.175981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 12:19:16.175981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:16.175981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 650/1283] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:16.181073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:16.277401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72d00c-a985-489f-9f6b-8e4f8c39a32a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:16.311344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:16.311941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:16.312215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:16.312482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:16.313146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:16.313446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:16.314803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:16.326711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 634/1284] 10.222.0.85 () {56 vars in 1109 bytes} [Wed Nov 2 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:16.332420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:16.431708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b78a3-3e2c-4138-be52-6cb113ad9aa1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:16.445590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:16.466383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:16.467014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:16.467286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:16.467552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:16.474390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:16.475811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 12:19:16.486948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:16.488050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Nov 02 12:19:16.489190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 651/1285] 10.222.0.85 () {56 vars in 1111 bytes} [Wed Nov 2 12:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:16.494076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:16.544115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:16.598664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854d091-eacd-4926-a97f-aed6439fc726 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:16.631929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:16.632524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:16.632795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:16.633087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:16.633770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:16.634067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:16.635479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634a910-c977-451b-a409-3ffc44d40f35 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:16.648294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 635/1286] 10.222.0.85 () {56 vars in 1148 bytes} [Wed Nov 2 12:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:16.654705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:16.758546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e6a5-5994-45a5-95b0-458bca7a6c50 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:16.791986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:16.792581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:16.792858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:16.793728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:16.794458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] RBAC: Adding query filter params (role.id=25bb4875811e4bacb09d3e72977a7b1f, scope.project.id=0c95872158724e60be029b048d666cf7, user.id=1de535a7ce1e475abdb02308cb3326e7) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:16.794760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:16.796174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614523d-d252-445d-8ed2-a655321f617c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:16.802037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 652/1287] 10.222.0.85 () {56 vars in 1374 bytes} [Wed Nov 2 12:19:16 2022] GET /identity/v3/role_assignments?user.id=1de535a7ce1e475abdb02308cb3326e7&scope.project.id=0c95872158724e60be029b048d666cf7&role.id=25bb4875811e4bacb09d3e72977a7b1f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:19:18.978771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6d790-83b5-4261-9563-f669f2003f6c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:18.979374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6d790-83b5-4261-9563-f669f2003f6c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:18.979374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6d790-83b5-4261-9563-f669f2003f6c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:18.981157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 636/1288] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:19:18.986586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:18.987044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:18.987134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:19.006050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:19.007267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:19.040455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:19.080074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 653/1289] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:19.346051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6d790-83b5-4261-9563-f669f2003f6c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:19.449013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6d790-83b5-4261-9563-f669f2003f6c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:19.460924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:19.485176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:19.485819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:19.486079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:19.486344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:19.486926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:19.488655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:19.504655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:19.505993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:19.507169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 637/1290] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:19:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:19.512274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:19.610592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9885d95-3423-4a96-a5b9-07113065d528 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:19.645923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:19.646521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:19.646901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:19.647366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:19.647743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:19.648262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:19.649487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f1d2d-1548-49cb-8ed1-7c581dfe1615 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:19.655819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 654/1291] 10.222.0.85 () {56 vars in 1116 bytes} [Wed Nov 2 12:19:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:19.661798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:19.783826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235177d7-0ed5-4e18-81b6-964af094cc2d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:19.820602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:19.821222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:19.821496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:19.821764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:19.822391 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] RBAC: Adding query filter params (type=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:19.822689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:19.824046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e1cf2-a7ab-48e2-8083-6f93535bfabe None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:19.828478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 638/1292] 10.222.0.85 () {56 vars in 1116 bytes} [Wed Nov 2 12:19:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:22.019051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bfe37-a834-416e-80a9-d3f1d788f1c7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:22.019642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bfe37-a834-416e-80a9-d3f1d788f1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:22.019642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bfe37-a834-416e-80a9-d3f1d788f1c7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:22.020990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 655/1293] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:19:22.028413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1ce11-9770-4a46-9e45-6c161c346691 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:22.028982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1ce11-9770-4a46-9e45-6c161c346691 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:22.029171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1ce11-9770-4a46-9e45-6c161c346691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:22.052580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f9d1ce11-9770-4a46-9e45-6c161c346691 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:22.053824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f9d1ce11-9770-4a46-9e45-6c161c346691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:22.085883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:22.130155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 639/1294] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:22.401374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bfe37-a834-416e-80a9-d3f1d788f1c7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:22.433268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:22.506306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bfe37-a834-416e-80a9-d3f1d788f1c7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:22.541287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63794f-e4ed-4850-8ba1-8fd6acb4040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:22.541901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63794f-e4ed-4850-8ba1-8fd6acb4040a None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:22.542175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63794f-e4ed-4850-8ba1-8fd6acb4040a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:22.542444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63794f-e4ed-4850-8ba1-8fd6acb4040a None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:22.543110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63794f-e4ed-4850-8ba1-8fd6acb4040a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:22.544480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63794f-e4ed-4850-8ba1-8fd6acb4040a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:22.560051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 656/1295] 10.222.0.85 () {60 vars in 1141 bytes} [Wed Nov 2 12:19:22 2022] POST /identity/v3/services => generated 246 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:19:24.801599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60e2ee-2f00-404d-a15d-d5bb1f8c8a90 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:24.802134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60e2ee-2f00-404d-a15d-d5bb1f8c8a90 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:24.802134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60e2ee-2f00-404d-a15d-d5bb1f8c8a90 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:24.803401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 640/1296] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:19:24.808990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:24.809528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:24.809822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:24.829400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:24.830660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:24.917558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 657/1297] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 12:19:25.187314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60e2ee-2f00-404d-a15d-d5bb1f8c8a90 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:25.305795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60e2ee-2f00-404d-a15d-d5bb1f8c8a90 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:25.343126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:25.343754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:25.344031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:25.344302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:25.344893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:25.346312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:25.355561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:25.357976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:25.359625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 641/1298] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:19:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:25.363979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:25.389304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:25.488711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b8bb-fbe4-4891-ac09-9bff41d42d6e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:25.527195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:25.528378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:25.528659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:25.528930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:25.529675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:25.529980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:25.531421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:25.537964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 658/1299] 10.222.0.85 () {56 vars in 1116 bytes} [Wed Nov 2 12:19:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:25.548452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:25.603258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:25.687685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7d2e7-f729-487a-885b-19615244e370 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:25.725068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:25.725603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:25.725880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:25.726148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:25.726856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] RBAC: Adding query filter params (interface=public, service_id=fd8279545e8f46e886a2b21320c87bad, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:25.727167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:25.728542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc5005-b536-4c7b-ac50-4fbce863f10e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:25.734406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 642/1300] 10.222.0.85 () {56 vars in 1251 bytes} [Wed Nov 2 12:19:25 2022] GET /identity/v3/endpoints?service_id=fd8279545e8f46e886a2b21320c87bad&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:25.743659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:25.829273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:25.848821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202d578-a369-469e-aab7-1617be135b02 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:25.884045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d2650-2d8d-4baa-a64f-865bc6736fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:25.884705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d2650-2d8d-4baa-a64f-865bc6736fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:25.885069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d2650-2d8d-4baa-a64f-865bc6736fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:25.885392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d2650-2d8d-4baa-a64f-865bc6736fe3 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:25.886103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d2650-2d8d-4baa-a64f-865bc6736fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:25.888277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d2650-2d8d-4baa-a64f-865bc6736fe3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:25.895482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 659/1301] 10.222.0.85 () {56 vars in 1087 bytes} [Wed Nov 2 12:19:25 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:19:28.110010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63801155-3525-4bc3-b069-730b4ee46856 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:28.110923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63801155-3525-4bc3-b069-730b4ee46856 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:28.110923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63801155-3525-4bc3-b069-730b4ee46856 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:28.111896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 643/1302] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:19:28.117522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:28.118253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:28.118412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:28.138441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:19:28.139644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:19:28.211118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 660/1303] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:19:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 12:19:28.481844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63801155-3525-4bc3-b069-730b4ee46856 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:28.589925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:28.598499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63801155-3525-4bc3-b069-730b4ee46856 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:28.636057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:28.637827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:28.637827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:28.638137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:28.638725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:28.640093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 12:19:28.649507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 12:19:28.651599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Nov 02 12:19:28.653245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 644/1304] 10.222.0.85 () {56 vars in 1117 bytes} [Wed Nov 2 12:19:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 12:19:28.657422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:28.737770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:28.763405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3694ae-9266-4a34-8ea7-5f50f21d6d2b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:28.801188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:28.801804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:28.802083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:28.802353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:28.802989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] RBAC: Adding query filter params (name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:19:28.803288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:28.804654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5d9bf-818a-4990-9889-357f7bdb0aca None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:28.810965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 661/1305] 10.222.0.85 () {56 vars in 1116 bytes} [Wed Nov 2 12:19:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:19:28.817168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:19:28.919263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08395054-ba0a-453d-9e76-e19c0f55b87a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:19:28.954362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70636f08-ccf9-45ad-850f-0baa95d7ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:19:28.954957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636f08-ccf9-45ad-850f-0baa95d7ccdb None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:19:28.955228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636f08-ccf9-45ad-850f-0baa95d7ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:19:28.955493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636f08-ccf9-45ad-850f-0baa95d7ccdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:19:28.956168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70636f08-ccf9-45ad-850f-0baa95d7ccdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:19:28.957558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70636f08-ccf9-45ad-850f-0baa95d7ccdb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:19:28.978581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 645/1306] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:19:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:19:38.807214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=30 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:19:38.973687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:36.852386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444607-bd8e-4ae5-a329-2122338acf57 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:36.853563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444607-bd8e-4ae5-a329-2122338acf57 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:36.854074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444607-bd8e-4ae5-a329-2122338acf57 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:36.860804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 662/1307] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:36.862389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c1db3-6d1d-4038-9fa1-f87c3d74e931 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:36.862916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c1db3-6d1d-4038-9fa1-f87c3d74e931 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:36.862916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c1db3-6d1d-4038-9fa1-f87c3d74e931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:36.961884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0f2c1db3-6d1d-4038-9fa1-f87c3d74e931 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:36.963128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f2c1db3-6d1d-4038-9fa1-f87c3d74e931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:37.146291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 646/1308] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:20:37.531292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f6778-4216-4f3f-840f-b25a36ac89d1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:37.531992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f6778-4216-4f3f-840f-b25a36ac89d1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:37.532128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f6778-4216-4f3f-840f-b25a36ac89d1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:37.533938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 663/1309] 10.222.0.85 () {54 vars in 964 bytes} [Wed Nov 2 12:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:37.542105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e455c5f-28d4-4ad2-a6f3-13d108174653 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:37.542707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e455c5f-28d4-4ad2-a6f3-13d108174653 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:37.543157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e455c5f-28d4-4ad2-a6f3-13d108174653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:37.644421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8e455c5f-28d4-4ad2-a6f3-13d108174653 None None] MFA Rules not processed for user `e102dacae430417aa150ddd53cf34f31`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:37.646113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8e455c5f-28d4-4ad2-a6f3-13d108174653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:37.670257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:37.743035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 647/1310] 10.222.0.85 () {58 vars in 1061 bytes} [Wed Nov 2 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:37.753990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f6778-4216-4f3f-840f-b25a36ac89d1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:37.756073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e38f6778-4216-4f3f-840f-b25a36ac89d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:37.887196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f6778-4216-4f3f-840f-b25a36ac89d1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:37.980500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:37.981619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:37.982090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:37.982567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:37.984664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:38.073047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:38.158084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:38.159582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09217ee-d0b5-4939-8ff2-8490819e0aef service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:38.187453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 664/1311] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:20:40.960234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3a371-5a11-4811-a9e4-c24b4f268c15 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:40.960824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3a371-5a11-4811-a9e4-c24b4f268c15 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:40.960824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3a371-5a11-4811-a9e4-c24b4f268c15 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:40.962170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 648/1312] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:40.968440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f72f8-432d-4a6f-b65e-45f663436284 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:40.968945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f72f8-432d-4a6f-b65e-45f663436284 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:40.969096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f72f8-432d-4a6f-b65e-45f663436284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:41.058138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c02f72f8-432d-4a6f-b65e-45f663436284 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:41.059373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c02f72f8-432d-4a6f-b65e-45f663436284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:41.166270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 665/1313] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:41.478912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3a371-5a11-4811-a9e4-c24b4f268c15 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:41.481072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ba3a371-5a11-4811-a9e4-c24b4f268c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:41.599451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3a371-5a11-4811-a9e4-c24b4f268c15 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:41.680885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:41.683876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:41.684555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:41.685332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:41.685959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:41.689079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:41.851270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:41.854006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac4a7e-f1a4-44d3-a053-8f9cb6b53a6b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:41.887866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 649/1314] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:20:44.421636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084e04e-8ea7-4709-ab56-33d1808ccac3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:44.422178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084e04e-8ea7-4709-ab56-33d1808ccac3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:44.422178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084e04e-8ea7-4709-ab56-33d1808ccac3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:44.423521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 666/1315] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:44.429083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c372f-9baf-421f-aad5-51e2dbf01728 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:44.429578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c372f-9baf-421f-aad5-51e2dbf01728 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:44.429733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c372f-9baf-421f-aad5-51e2dbf01728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:44.499724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8e1c372f-9baf-421f-aad5-51e2dbf01728 None None] MFA Rules not processed for user `c93deef3b5ac4785afdd5c6bf9559b40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:44.500942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8e1c372f-9baf-421f-aad5-51e2dbf01728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:44.562683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:44.612399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 650/1316] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:44.901621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084e04e-8ea7-4709-ab56-33d1808ccac3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:44.903190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8084e04e-8ea7-4709-ab56-33d1808ccac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:44.913623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:45.013906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084e04e-8ea7-4709-ab56-33d1808ccac3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:45.103409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:45.104037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:45.104307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:45.104581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:45.106886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:45.177216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:45.230421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:45.231885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552c183-f2bb-47a9-813f-226a478b5cf4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:45.253351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 667/1317] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:20:48.071516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e1974-8e23-4c3d-92b2-37efc21e18c5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:48.072063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e1974-8e23-4c3d-92b2-37efc21e18c5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:48.072063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e1974-8e23-4c3d-92b2-37efc21e18c5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:48.074050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 651/1318] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:48.080045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795ad7c-e3a6-4126-a8a3-91c3948fa324 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:48.080600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795ad7c-e3a6-4126-a8a3-91c3948fa324 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:48.080600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795ad7c-e3a6-4126-a8a3-91c3948fa324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:48.156053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a795ad7c-e3a6-4126-a8a3-91c3948fa324 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:48.157345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a795ad7c-e3a6-4126-a8a3-91c3948fa324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:48.259718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 668/1319] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:48.552575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e1974-8e23-4c3d-92b2-37efc21e18c5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:48.554195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cf8e1974-8e23-4c3d-92b2-37efc21e18c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:48.678398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e1974-8e23-4c3d-92b2-37efc21e18c5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:48.764859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:48.765413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:48.765683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:48.765950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:48.767579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:48.775116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:48.884165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:48.885647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af537c3d-6439-4695-a095-dfa274c775ed service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:48.904960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 652/1320] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:20:51.922172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe7e9-d9e9-4fd2-a10b-036b001559cc None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:51.922708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe7e9-d9e9-4fd2-a10b-036b001559cc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:51.922708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe7e9-d9e9-4fd2-a10b-036b001559cc None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:51.923969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 669/1321] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:51.930460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e89fa-7759-4bf7-a86f-b57c56986d93 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:51.931080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e89fa-7759-4bf7-a86f-b57c56986d93 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:51.931080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e89fa-7759-4bf7-a86f-b57c56986d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:52.020880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a47e89fa-7759-4bf7-a86f-b57c56986d93 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:52.022190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a47e89fa-7759-4bf7-a86f-b57c56986d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:52.095999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:52.147209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 653/1322] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:52.435889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe7e9-d9e9-4fd2-a10b-036b001559cc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:52.437546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4ebe7e9-d9e9-4fd2-a10b-036b001559cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:52.461250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:52.583768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe7e9-d9e9-4fd2-a10b-036b001559cc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:52.689928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:52.690647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:52.690974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:52.691295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:52.693279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:52.795155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:52.854981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:52.857633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbbad4-86ea-4475-8160-e3770c130ae5 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:52.883211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 670/1323] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:20:55.553485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0059e9e-825a-4b0a-a814-22b51a948fea None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:55.554019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0059e9e-825a-4b0a-a814-22b51a948fea None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:55.554019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0059e9e-825a-4b0a-a814-22b51a948fea None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:55.555378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 654/1324] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:55.562017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1199-0130-42a3-8475-689f3a144bf8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:55.562499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1199-0130-42a3-8475-689f3a144bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:55.562822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1199-0130-42a3-8475-689f3a144bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:55.645361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e46f1199-0130-42a3-8475-689f3a144bf8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:55.646772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e46f1199-0130-42a3-8475-689f3a144bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:55.773743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 671/1325] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:56.082895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf976ad-6f58-42d5-a0a0-7cb0fca7d623 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:56.083279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf976ad-6f58-42d5-a0a0-7cb0fca7d623 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:56.083554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf976ad-6f58-42d5-a0a0-7cb0fca7d623 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:56.084932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 655/1326] 10.222.0.85 () {54 vars in 964 bytes} [Wed Nov 2 12:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:20:56.092209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b536f0-0b67-40ef-81fe-5da52d720b0d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:56.092484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b536f0-0b67-40ef-81fe-5da52d720b0d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:56.092749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b536f0-0b67-40ef-81fe-5da52d720b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:56.130056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:56.177766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-30b536f0-0b67-40ef-81fe-5da52d720b0d None None] MFA Rules not processed for user `e102dacae430417aa150ddd53cf34f31`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:56.178984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30b536f0-0b67-40ef-81fe-5da52d720b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:56.260218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 672/1327] 10.222.0.85 () {58 vars in 1061 bytes} [Wed Nov 2 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:56.270638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf976ad-6f58-42d5-a0a0-7cb0fca7d623 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:56.272274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4cf976ad-6f58-42d5-a0a0-7cb0fca7d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:56.392005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf976ad-6f58-42d5-a0a0-7cb0fca7d623 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:56.476659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:56.477300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:56.477579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:56.477845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:56.480018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:56.498976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:56.603260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:56.611216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248f699-eeb7-421b-8895-217b9afed3ce service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:56.629020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 656/1328] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:20:59.134869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ccfd0-6d3a-4e11-b063-80a4304c7089 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:59.135404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ccfd0-6d3a-4e11-b063-80a4304c7089 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:59.135404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ccfd0-6d3a-4e11-b063-80a4304c7089 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:59.137213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 673/1329] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:20:59.143441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97543d2-0c9b-4837-945f-cf953f542289 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:59.144039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97543d2-0c9b-4837-945f-cf953f542289 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:59.144039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97543d2-0c9b-4837-945f-cf953f542289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:59.215885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a97543d2-0c9b-4837-945f-cf953f542289 None None] MFA Rules not processed for user `c93deef3b5ac4785afdd5c6bf9559b40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:20:59.217152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a97543d2-0c9b-4837-945f-cf953f542289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:59.300274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:59.338465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 657/1330] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:20:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:20:59.637346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ccfd0-6d3a-4e11-b063-80a4304c7089 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:20:59.639197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a06ccfd0-6d3a-4e11-b063-80a4304c7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:59.750726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ccfd0-6d3a-4e11-b063-80a4304c7089 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:20:59.774547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:20:59.840625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:20:59.841613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:20:59.841901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:20:59.842174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:20:59.843849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:20:59.959360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:20:59.960828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2312915-2e7e-4c89-ac7f-0d237fa2125b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:20:59.982370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 674/1331] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:20:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:02.707092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830a91-b92f-4bbe-a08f-57d95933d3c6 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:02.707697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830a91-b92f-4bbe-a08f-57d95933d3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:02.708040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830a91-b92f-4bbe-a08f-57d95933d3c6 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:02.710517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 658/1332] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:02.716075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7610c-02be-488e-9578-e581c0a46d41 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:02.716584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7610c-02be-488e-9578-e581c0a46d41 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:02.716697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7610c-02be-488e-9578-e581c0a46d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:02.786867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:02.789670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-86b7610c-02be-488e-9578-e581c0a46d41 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:02.792193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-86b7610c-02be-488e-9578-e581c0a46d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:02.899535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 675/1333] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:03.201344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74830a91-b92f-4bbe-a08f-57d95933d3c6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:03.202906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74830a91-b92f-4bbe-a08f-57d95933d3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:03.313240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74830a91-b92f-4bbe-a08f-57d95933d3c6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:03.392201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:03.392804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:03.393109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:03.393382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:03.395036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:03.424719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:03.518660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:03.520100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adde0fe-4822-4155-ab06-b6f026817e71 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:03.540687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 659/1334] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:05.816074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07cb25-b406-4857-8cef-1b549e1a9bad None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:05.816632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07cb25-b406-4857-8cef-1b549e1a9bad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:05.816632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07cb25-b406-4857-8cef-1b549e1a9bad None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:05.818058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 676/1335] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:21:05.824775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de622-d473-4aa8-a03b-5c4c4d39deae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:05.825412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de622-d473-4aa8-a03b-5c4c4d39deae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:05.825412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de622-d473-4aa8-a03b-5c4c4d39deae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:05.903304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b92de622-d473-4aa8-a03b-5c4c4d39deae None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:05.904542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b92de622-d473-4aa8-a03b-5c4c4d39deae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:05.990500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:06.030138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 660/1336] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:21:06.313728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07cb25-b406-4857-8cef-1b549e1a9bad None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:06.315286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5a07cb25-b406-4857-8cef-1b549e1a9bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:06.426415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07cb25-b406-4857-8cef-1b549e1a9bad None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:06.468468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:06.524798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:06.525444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:06.525723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:06.525994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:06.527608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:06.648539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:06.650063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d06ed5-d245-4f21-aee6-33dc8ba9fe0b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:06.670679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 677/1337] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:21:09.325653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92e4e-394f-4eab-ad30-9fe43fd999b9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:09.326498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92e4e-394f-4eab-ad30-9fe43fd999b9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:09.326689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92e4e-394f-4eab-ad30-9fe43fd999b9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:09.328058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 661/1338] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:09.342132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6104-5874-48e6-bc9f-200e2b317c7e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:09.342688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6104-5874-48e6-bc9f-200e2b317c7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:09.343134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6104-5874-48e6-bc9f-200e2b317c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:09.417926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8fff6104-5874-48e6-bc9f-200e2b317c7e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:09.419155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8fff6104-5874-48e6-bc9f-200e2b317c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:09.427678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:09.537960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 678/1339] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:09.823941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea92e4e-394f-4eab-ad30-9fe43fd999b9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:09.826057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ea92e4e-394f-4eab-ad30-9fe43fd999b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:09.956974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea92e4e-394f-4eab-ad30-9fe43fd999b9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:10.034596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:10.035207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:10.035480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:10.035743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:10.037377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:10.075696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:10.164979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:10.168012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a615-e295-4f4d-9d16-0371c9d2ccf2 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:10.194666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 662/1340] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:14.547821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6426-a619-4e52-ae8d-08938a79af52 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:14.548268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6426-a619-4e52-ae8d-08938a79af52 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:14.548453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6426-a619-4e52-ae8d-08938a79af52 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:14.549654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 679/1341] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:14.555976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619bbdf-1d09-4bb7-94ff-c3bea41e8bdd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:14.556527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619bbdf-1d09-4bb7-94ff-c3bea41e8bdd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:14.556962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619bbdf-1d09-4bb7-94ff-c3bea41e8bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:14.648955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d619bbdf-1d09-4bb7-94ff-c3bea41e8bdd None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:14.650316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d619bbdf-1d09-4bb7-94ff-c3bea41e8bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:14.763898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:14.777096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 663/1342] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:15.129777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a6426-a619-4e52-ae8d-08938a79af52 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:15.131461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-280a6426-a619-4e52-ae8d-08938a79af52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:15.242111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a6426-a619-4e52-ae8d-08938a79af52 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:15.281930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:15.321305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:15.321912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:15.322202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:15.322466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:15.324074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:15.450787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:15.452221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09d61b-586f-4759-89c9-33b000ba4ae9 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:15.474613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 680/1343] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:18.391216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229399-b730-4bd7-a07e-123f2d067743 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:18.391730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229399-b730-4bd7-a07e-123f2d067743 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:18.391730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229399-b730-4bd7-a07e-123f2d067743 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:18.393023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 664/1344] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:18.398210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632e99-576c-45e4-8b98-08913a79e88a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:18.398643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632e99-576c-45e4-8b98-08913a79e88a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:18.398740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632e99-576c-45e4-8b98-08913a79e88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:18.468135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1d632e99-576c-45e4-8b98-08913a79e88a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:18.469406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1d632e99-576c-45e4-8b98-08913a79e88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:18.488771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:18.571032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 681/1345] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:18.855839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab229399-b730-4bd7-a07e-123f2d067743 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:18.857403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab229399-b730-4bd7-a07e-123f2d067743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:18.973292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab229399-b730-4bd7-a07e-123f2d067743 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:19.058030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:19.058703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:19.058982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:19.059253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:19.061053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:19.118421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:19.198336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:19.199891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee9362-d3ee-4249-b9de-9b80da0d02fd service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:19.222064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 665/1346] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:23.505066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4902cb2-e618-4e9f-aa6f-a264f0af5347 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:23.505568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4902cb2-e618-4e9f-aa6f-a264f0af5347 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:23.505664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4902cb2-e618-4e9f-aa6f-a264f0af5347 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:23.507129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 682/1347] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:23.512524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e47d9-d7dc-4103-ba87-cd1ff5ba6dfe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:23.513352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e47d9-d7dc-4103-ba87-cd1ff5ba6dfe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:23.513352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e47d9-d7dc-4103-ba87-cd1ff5ba6dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:23.586499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-759e47d9-d7dc-4103-ba87-cd1ff5ba6dfe None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:23.587723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-759e47d9-d7dc-4103-ba87-cd1ff5ba6dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:23.687936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:23.690592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 666/1348] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:23.977435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4902cb2-e618-4e9f-aa6f-a264f0af5347 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:23.979111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4902cb2-e618-4e9f-aa6f-a264f0af5347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:24.086193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4902cb2-e618-4e9f-aa6f-a264f0af5347 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:24.136958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:24.163192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:24.163796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:24.164073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:24.164347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:24.166020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:24.277242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:24.278615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbc165-afee-4921-a661-b0071593bafc service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:24.297312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 683/1349] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:29.464070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50e23a-a95b-4022-88a2-65fa6f26daa0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:29.464571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50e23a-a95b-4022-88a2-65fa6f26daa0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:29.464681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50e23a-a95b-4022-88a2-65fa6f26daa0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:29.468303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 667/1350] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:29.476104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303bae-6028-4601-a76e-87cb81937848 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:29.479317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303bae-6028-4601-a76e-87cb81937848 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:29.479317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303bae-6028-4601-a76e-87cb81937848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:29.594161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5e303bae-6028-4601-a76e-87cb81937848 None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:29.595657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5e303bae-6028-4601-a76e-87cb81937848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:29.619328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:29.761284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 684/1351] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:34.085329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b248c8-410f-46e2-8408-05406bcd9502 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:34.085890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b248c8-410f-46e2-8408-05406bcd9502 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:34.085890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b248c8-410f-46e2-8408-05406bcd9502 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:34.087399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 668/1352] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:34.093036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972002d-9eeb-4220-af94-215be13fb3bc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:34.093506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972002d-9eeb-4220-af94-215be13fb3bc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:34.093765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972002d-9eeb-4220-af94-215be13fb3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:34.182339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0972002d-9eeb-4220-af94-215be13fb3bc None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:34.183994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0972002d-9eeb-4220-af94-215be13fb3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:34.264429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:34.302449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 685/1353] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:35.960902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13da2b-d2e4-4b3d-9552-8a8e61eda0d7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:35.961484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13da2b-d2e4-4b3d-9552-8a8e61eda0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:35.961484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13da2b-d2e4-4b3d-9552-8a8e61eda0d7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:35.963384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 669/1354] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:35.968847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb6e8-a560-4a97-b10f-a9714df13a78 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:35.969358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb6e8-a560-4a97-b10f-a9714df13a78 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:35.969478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb6e8-a560-4a97-b10f-a9714df13a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:36.050960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-468bb6e8-a560-4a97-b10f-a9714df13a78 None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:36.052215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-468bb6e8-a560-4a97-b10f-a9714df13a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:36.172345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 686/1355] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:39.635035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5def7-c490-43c1-a8b0-3cfb26077460 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:39.635989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5def7-c490-43c1-a8b0-3cfb26077460 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:39.636488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5def7-c490-43c1-a8b0-3cfb26077460 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:39.644211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 670/1356] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:39.644467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e5afc-d821-4cf9-8bd6-2267186bda26 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:39.644939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e5afc-d821-4cf9-8bd6-2267186bda26 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:39.645536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e5afc-d821-4cf9-8bd6-2267186bda26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:39.691241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:39.733938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2c2e5afc-d821-4cf9-8bd6-2267186bda26 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:39.735074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c2e5afc-d821-4cf9-8bd6-2267186bda26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:39.849114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 687/1357] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:40.147572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274978-62c8-404b-afb5-08620ac3d0ec None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:40.148665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274978-62c8-404b-afb5-08620ac3d0ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:40.149200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274978-62c8-404b-afb5-08620ac3d0ec None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:40.151259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 671/1358] 10.222.0.85 () {54 vars in 960 bytes} [Wed Nov 2 12:21:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:40.159970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077df22-980e-4336-918c-89db26fb65a6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:40.160847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077df22-980e-4336-918c-89db26fb65a6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:40.161869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077df22-980e-4336-918c-89db26fb65a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:40.253554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1077df22-980e-4336-918c-89db26fb65a6 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:40.254775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1077df22-980e-4336-918c-89db26fb65a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:40.314902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:40.394933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 688/1359] 10.222.0.85 () {58 vars in 1057 bytes} [Wed Nov 2 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:40.403497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274978-62c8-404b-afb5-08620ac3d0ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:40.405157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1a274978-62c8-404b-afb5-08620ac3d0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:40.585741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274978-62c8-404b-afb5-08620ac3d0ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:40.669755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:40.716987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:40.717758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:40.718094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:40.718432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:40.720067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:40.835393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:40.836684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0237d-e224-484f-9bbd-b82d7b915fab service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:40.858862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 672/1360] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:43.267869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36104580-6e84-418b-988a-d72fcb42aaf7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:43.268495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36104580-6e84-418b-988a-d72fcb42aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:43.268495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36104580-6e84-418b-988a-d72fcb42aaf7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:43.270393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 689/1361] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:43.280835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b70d1-cfe0-47b5-8bb6-de73ef3353fe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:43.280835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b70d1-cfe0-47b5-8bb6-de73ef3353fe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:43.280835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b70d1-cfe0-47b5-8bb6-de73ef3353fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:43.367717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ff9b70d1-cfe0-47b5-8bb6-de73ef3353fe None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:43.370820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ff9b70d1-cfe0-47b5-8bb6-de73ef3353fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:43.419774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:43.488498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 673/1362] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:43.783286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36104580-6e84-418b-988a-d72fcb42aaf7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:43.785042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-36104580-6e84-418b-988a-d72fcb42aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:43.975854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36104580-6e84-418b-988a-d72fcb42aaf7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:43.987004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:44.111009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:44.111656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:44.112309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:44.112309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:44.113976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:44.215909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:44.217523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bd9db-f92c-45a1-9040-01831fc084c6 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:44.237338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 690/1363] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:46.778277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d0df-1ea2-4eb7-ad88-aa5674734453 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:46.778812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d0df-1ea2-4eb7-ad88-aa5674734453 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:46.778812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d0df-1ea2-4eb7-ad88-aa5674734453 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:46.780140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 674/1364] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:46.785643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7bb45-c5aa-4a42-8ade-9d4d02fb4048 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:46.786128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7bb45-c5aa-4a42-8ade-9d4d02fb4048 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:46.786208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7bb45-c5aa-4a42-8ade-9d4d02fb4048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:46.803137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:46.869569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-92d7bb45-c5aa-4a42-8ade-9d4d02fb4048 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:46.870920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92d7bb45-c5aa-4a42-8ade-9d4d02fb4048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:46.987074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 691/1365] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:47.277880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3449c87-2c56-4289-9758-4cf7c6cfee2d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:47.278153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3449c87-2c56-4289-9758-4cf7c6cfee2d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:47.278596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3449c87-2c56-4289-9758-4cf7c6cfee2d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:47.280071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 675/1366] 10.222.0.85 () {54 vars in 960 bytes} [Wed Nov 2 12:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:47.285667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf83ae-9a60-487b-8d52-1c3da9128579 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:47.285948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf83ae-9a60-487b-8d52-1c3da9128579 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:47.286215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf83ae-9a60-487b-8d52-1c3da9128579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:47.367088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e1cf83ae-9a60-487b-8d52-1c3da9128579 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:47.368286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e1cf83ae-9a60-487b-8d52-1c3da9128579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:47.407410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:47.516736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 692/1367] 10.222.0.85 () {58 vars in 1057 bytes} [Wed Nov 2 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:47.528632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3449c87-2c56-4289-9758-4cf7c6cfee2d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:47.530543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3449c87-2c56-4289-9758-4cf7c6cfee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:47.732577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3449c87-2c56-4289-9758-4cf7c6cfee2d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:47.755418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:47.859726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:47.860647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:47.860647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:47.860796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:47.862425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:47.981470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:47.983100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9185c9-9569-4a45-8b69-0944c60655b6 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:48.004307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 676/1368] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:48.023271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f92280-2456-4f8a-8e0b-6ff21ee30708 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:48.023833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f92280-2456-4f8a-8e0b-6ff21ee30708 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:48.023833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f92280-2456-4f8a-8e0b-6ff21ee30708 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:48.025763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 693/1369] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:48.031553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84b9c9-cfdd-4f27-a605-d17ca62b8091 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:48.031834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84b9c9-cfdd-4f27-a605-d17ca62b8091 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:48.032107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84b9c9-cfdd-4f27-a605-d17ca62b8091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:48.076287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:48.127772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ca84b9c9-cfdd-4f27-a605-d17ca62b8091 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:48.129416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca84b9c9-cfdd-4f27-a605-d17ca62b8091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:48.262371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 677/1370] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:48.604809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d8bc6-6ab4-4f4a-b3fa-1d543d25638e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:48.604809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d8bc6-6ab4-4f4a-b3fa-1d543d25638e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:48.604809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d8bc6-6ab4-4f4a-b3fa-1d543d25638e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:48.606615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 694/1371] 10.222.0.85 () {54 vars in 962 bytes} [Wed Nov 2 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:48.610808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376dcbb-bcf4-4954-bf80-960d168a78b1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:48.611235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376dcbb-bcf4-4954-bf80-960d168a78b1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:48.611607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376dcbb-bcf4-4954-bf80-960d168a78b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:48.702240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8376dcbb-bcf4-4954-bf80-960d168a78b1 None None] MFA Rules not processed for user `b840491b8d05489c8475e1125dbc0b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:48.703268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8376dcbb-bcf4-4954-bf80-960d168a78b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:48.761273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:48.799279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 678/1372] 10.222.0.85 () {58 vars in 1059 bytes} [Wed Nov 2 12:21:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:48.807312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d8bc6-6ab4-4f4a-b3fa-1d543d25638e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:48.809099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-512d8bc6-6ab4-4f4a-b3fa-1d543d25638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:48.934552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d8bc6-6ab4-4f4a-b3fa-1d543d25638e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:48.982046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:49.033359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:49.034075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:49.034356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:49.034642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:49.036400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:49.181548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:49.182829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fb3aa-4341-4f3b-8781-3ee0436717bc service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:49.203300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 695/1373] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:21:50.294347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7a8f3-36cf-4950-a60a-96bf8e51d59a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:50.294885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7a8f3-36cf-4950-a60a-96bf8e51d59a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:50.294885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7a8f3-36cf-4950-a60a-96bf8e51d59a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:50.296216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 679/1374] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:50.301614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7113-5a96-4a57-93cc-3bb44c98c45e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:50.302089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7113-5a96-4a57-93cc-3bb44c98c45e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:50.302209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7113-5a96-4a57-93cc-3bb44c98c45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:50.373167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca7e7113-5a96-4a57-93cc-3bb44c98c45e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:50.374562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca7e7113-5a96-4a57-93cc-3bb44c98c45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:50.396795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:50.480708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 696/1375] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:50.772589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7a8f3-36cf-4950-a60a-96bf8e51d59a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:50.774346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-44f7a8f3-36cf-4950-a60a-96bf8e51d59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:50.953678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7a8f3-36cf-4950-a60a-96bf8e51d59a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:51.045909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:51.126272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:51.126885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:51.127161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:51.127528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:51.129117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:51.236064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:51.237674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9af05-0db6-41a8-9467-c9c2f4b0223f service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:51.257551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 680/1376] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:53.511167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c34af-c325-4f0d-b4f6-54bf47e0b527 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:53.511736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c34af-c325-4f0d-b4f6-54bf47e0b527 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:53.511736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c34af-c325-4f0d-b4f6-54bf47e0b527 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:53.513498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 697/1377] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:53.519267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a58c4-88d5-46dc-8be7-fd78d6e6260f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:53.519765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a58c4-88d5-46dc-8be7-fd78d6e6260f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:53.519864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a58c4-88d5-46dc-8be7-fd78d6e6260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:53.599491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ab0a58c4-88d5-46dc-8be7-fd78d6e6260f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:53.600782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab0a58c4-88d5-46dc-8be7-fd78d6e6260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:53.610160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:53.722049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 681/1378] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:54.019838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c34af-c325-4f0d-b4f6-54bf47e0b527 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:54.021390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b06c34af-c325-4f0d-b4f6-54bf47e0b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:54.173742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c34af-c325-4f0d-b4f6-54bf47e0b527 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:54.177492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:54.312075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:54.313926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:54.313926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:54.313926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:54.315278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:54.419878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:54.421506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d6b29-f930-47e4-bf2a-938846330cff service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:54.444433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 698/1379] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:21:56.719099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bf95f-a642-47f3-b687-2e640508f4eb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:56.719583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bf95f-a642-47f3-b687-2e640508f4eb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:56.719702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bf95f-a642-47f3-b687-2e640508f4eb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:56.722204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 682/1380] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:21:56.726760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdafe0-25e7-4b1d-9237-fa44ebc64783 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:56.726760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdafe0-25e7-4b1d-9237-fa44ebc64783 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:56.728179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdafe0-25e7-4b1d-9237-fa44ebc64783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:56.740937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:56.836514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-47bdafe0-25e7-4b1d-9237-fa44ebc64783 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:21:56.839059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-47bdafe0-25e7-4b1d-9237-fa44ebc64783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:56.963626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 699/1381] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:21:57.313722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bf95f-a642-47f3-b687-2e640508f4eb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:21:57.315229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-201bf95f-a642-47f3-b687-2e640508f4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:57.473970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:57.490179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bf95f-a642-47f3-b687-2e640508f4eb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:21:57.648258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:21:57.648775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:21:57.649137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:21:57.650114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:21:57.651507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:21:57.762709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:21:57.764568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09735-c055-4641-8a47-8c7381147dff service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:21:57.775967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:21:57.785051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 683/1382] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:00.442567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f9e34-e4d9-44e2-a782-fa2762397a70 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:00.443176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f9e34-e4d9-44e2-a782-fa2762397a70 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:00.443849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f9e34-e4d9-44e2-a782-fa2762397a70 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:00.446629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 700/1383] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:00.452146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7b4e2-5f48-405c-a22c-ca613d8ac181 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:00.452574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7b4e2-5f48-405c-a22c-ca613d8ac181 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:00.452688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7b4e2-5f48-405c-a22c-ca613d8ac181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:00.528615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-32b7b4e2-5f48-405c-a22c-ca613d8ac181 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:00.529886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-32b7b4e2-5f48-405c-a22c-ca613d8ac181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:00.651968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 684/1384] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:01.003630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f9e34-e4d9-44e2-a782-fa2762397a70 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:01.006238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b90f9e34-e4d9-44e2-a782-fa2762397a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:01.123012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:01.196361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f9e34-e4d9-44e2-a782-fa2762397a70 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:01.315137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:01.315643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:01.316081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:01.316406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:01.318153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:01.376417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:01.420697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:01.422342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcb4eb-76e4-407d-8317-f1cd15682772 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:01.441144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 701/1385] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:03.890721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ca43-e566-45a3-8aad-16e597eaf308 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:03.891273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ca43-e566-45a3-8aad-16e597eaf308 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:03.891273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ca43-e566-45a3-8aad-16e597eaf308 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:03.892794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 685/1386] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:22:03.898466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10533bfc-a8e9-4f7b-a361-74903e65e86c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:03.899029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10533bfc-a8e9-4f7b-a361-74903e65e86c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:03.899029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10533bfc-a8e9-4f7b-a361-74903e65e86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:03.975211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-10533bfc-a8e9-4f7b-a361-74903e65e86c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:03.976450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-10533bfc-a8e9-4f7b-a361-74903e65e86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:04.090601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 702/1387] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:04.383934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ca43-e566-45a3-8aad-16e597eaf308 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:04.385566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1db4ca43-e566-45a3-8aad-16e597eaf308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:04.471521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:04.545451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ca43-e566-45a3-8aad-16e597eaf308 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:04.665059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:04.665960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:04.666640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:04.666640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:04.668068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:04.708696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:04.766227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:04.767889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251095e-e13c-4b2e-9c82-aea82abcc1d4 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:04.788654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 686/1388] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:04.880518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0eaa1-5b93-4876-bc5d-a03a94c36b82 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:04.880783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0eaa1-5b93-4876-bc5d-a03a94c36b82 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:04.881095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0eaa1-5b93-4876-bc5d-a03a94c36b82 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:04.882846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 703/1389] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:04.889198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d0060-89b2-4aca-a3f7-4812f79f514f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:04.889481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d0060-89b2-4aca-a3f7-4812f79f514f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:04.889755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d0060-89b2-4aca-a3f7-4812f79f514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:04.970330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8f8d0060-89b2-4aca-a3f7-4812f79f514f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:04.971595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8f8d0060-89b2-4aca-a3f7-4812f79f514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:05.080020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 687/1390] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:22:05.379187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ba52-2160-420b-883b-e866306260c8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:05.379679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ba52-2160-420b-883b-e866306260c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:05.379859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ba52-2160-420b-883b-e866306260c8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:05.381284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 704/1391] 10.222.0.85 () {54 vars in 962 bytes} [Wed Nov 2 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:05.388190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4f7f-c5e7-451c-bcae-afa07f068e73 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:05.388474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4f7f-c5e7-451c-bcae-afa07f068e73 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:05.388744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4f7f-c5e7-451c-bcae-afa07f068e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:05.405619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:05.469848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-71af4f7f-c5e7-451c-bcae-afa07f068e73 None None] MFA Rules not processed for user `ce6a8adf15e3498d95442b3ad0ef3864`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:05.471057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-71af4f7f-c5e7-451c-bcae-afa07f068e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:05.547585 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 688/1392] 10.222.0.85 () {58 vars in 1059 bytes} [Wed Nov 2 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:05.557328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ba52-2160-420b-883b-e866306260c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:05.558761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2b00ba52-2160-420b-883b-e866306260c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:05.595047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:05.678608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ba52-2160-420b-883b-e866306260c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:05.761414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:05.762018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:05.762299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:05.762569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:05.764443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:05.846621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:05.879864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:05.881314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03e9af-d4c1-45ef-9dc4-736cc50800f4 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:05.900758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 705/1393] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:06.658925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351071b-16dd-4c37-b19a-589b8967440a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:06.659415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351071b-16dd-4c37-b19a-589b8967440a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:06.659508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351071b-16dd-4c37-b19a-589b8967440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:06.738975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3351071b-16dd-4c37-b19a-589b8967440a None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:06.740196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3351071b-16dd-4c37-b19a-589b8967440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:06.801911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:06.837345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 689/1394] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:06.882467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7654a6-e17a-407b-b255-83660ca65af4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:06.883021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7654a6-e17a-407b-b255-83660ca65af4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:06.883021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7654a6-e17a-407b-b255-83660ca65af4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:06.884577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 706/1395] 10.222.0.85 () {54 vars in 943 bytes} [Wed Nov 2 12:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:06.893729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:06.894150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:06.894429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:06.984842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] MFA Rules not processed for user `ef33e7dcb56a481aa051c09d45bfc83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:06.986101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:07.033027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72f2bf-f487-4504-ad27-f95afa3b468b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:07.033388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72f2bf-f487-4504-ad27-f95afa3b468b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:07.033658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72f2bf-f487-4504-ad27-f95afa3b468b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:07.035114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 707/1396] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:22:07.040907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:07.041294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:07.041639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:07.104745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 690/1397] 10.222.0.85 () {58 vars in 1040 bytes} [Wed Nov 2 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:07.113183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:07.114220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:07.114446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:07.124672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a75f9-1660-4c15-a20b-bdcabac54f97 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:07.154239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:07.235329 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 708/1398] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:07.245697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:07.246318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:07.246596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:07.246865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:07.248651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:07.345344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:07.346987 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0b0b4-2709-455f-b374-91480a54ca9a service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:07.356137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 691/1399] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:22:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 12:22:07.533047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:07.537990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:07.584426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:07.678174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dafe33-399a-4194-b220-82ed6eba8b80 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:07.936696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:07.937409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:07.937690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:07.937810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:07.939444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:07.980873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:08.050368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:08.052032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd8dad-606f-4094-84f7-fdddfc1e9e0f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:08.074799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 709/1400] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:10.338716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf680d7b-a78e-4725-bfa5-0a799f4faa90 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:10.339303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf680d7b-a78e-4725-bfa5-0a799f4faa90 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:10.339303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf680d7b-a78e-4725-bfa5-0a799f4faa90 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:10.340703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 692/1401] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:10.346824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b15fb-e75c-4a45-aff2-5b22b449453c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:10.347406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b15fb-e75c-4a45-aff2-5b22b449453c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:10.347406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b15fb-e75c-4a45-aff2-5b22b449453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:10.430805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ee5b15fb-e75c-4a45-aff2-5b22b449453c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:10.432064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ee5b15fb-e75c-4a45-aff2-5b22b449453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:10.546481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 710/1402] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:10.858039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf680d7b-a78e-4725-bfa5-0a799f4faa90 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:10.859567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bf680d7b-a78e-4725-bfa5-0a799f4faa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:10.934337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:11.031119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf680d7b-a78e-4725-bfa5-0a799f4faa90 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:11.164953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:11.165990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:11.166344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:11.166616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:11.168385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:11.191571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:11.275748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:11.277406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3f392-4632-439c-9ca4-0155a91e82d7 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:11.298691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 693/1403] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:13.553444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58001d0-853b-4516-beab-d769a5d2feac None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:13.553444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58001d0-853b-4516-beab-d769a5d2feac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:13.553444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58001d0-853b-4516-beab-d769a5d2feac None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:13.558791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de939bec-3bcb-4deb-9782-3f0b39808d04 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:13.559603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de939bec-3bcb-4deb-9782-3f0b39808d04 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:13.560091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de939bec-3bcb-4deb-9782-3f0b39808d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:13.564803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 711/1404] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:13 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:13.640066 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-de939bec-3bcb-4deb-9782-3f0b39808d04 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:13.641409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-de939bec-3bcb-4deb-9782-3f0b39808d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:13.753033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:13.761375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 694/1405] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:22:14.063646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58001d0-853b-4516-beab-d769a5d2feac None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:14.065547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e58001d0-853b-4516-beab-d769a5d2feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:14.077187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:14.232191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58001d0-853b-4516-beab-d769a5d2feac None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:14.331355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:14.368013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:14.368618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:14.368887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:14.369183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:14.370901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:14.475959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:14.477809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608d255-1bde-4b00-aa8b-81b302110404 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:14.501599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 712/1406] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:16.815843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d7397-92f8-4c92-a834-a7931976d320 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:16.816748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d7397-92f8-4c92-a834-a7931976d320 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:16.817047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d7397-92f8-4c92-a834-a7931976d320 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:16.818675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 695/1407] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:16.824034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d03f-bed1-44ac-959b-24b14a25f3db None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:16.824508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d03f-bed1-44ac-959b-24b14a25f3db None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:16.824625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d03f-bed1-44ac-959b-24b14a25f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:16.898894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1466d03f-bed1-44ac-959b-24b14a25f3db None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:16.900123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1466d03f-bed1-44ac-959b-24b14a25f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:16.954065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:17.015534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 713/1408] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:17.310387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d7397-92f8-4c92-a834-a7931976d320 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:17.311917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5b2d7397-92f8-4c92-a834-a7931976d320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:17.481412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d7397-92f8-4c92-a834-a7931976d320 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:17.566669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:17.622965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:17.623571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:17.623841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:17.624123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:17.625830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:17.725657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:17.727329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea988c8-80bd-4430-9f09-5473b8a4900b service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:17.747089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 696/1409] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:27.013958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=23 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:27.744814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=64 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:30.679900 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6227b-ec46-4701-a2fd-fe00b832adf7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:30.680466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6227b-ec46-4701-a2fd-fe00b832adf7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:30.680466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6227b-ec46-4701-a2fd-fe00b832adf7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:30.681767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 714/1410] 10.222.0.85 () {54 vars in 918 bytes} [Wed Nov 2 12:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:30.687088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3200e4e-696b-434b-8403-f6a1d8b6dcd1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:30.687612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3200e4e-696b-434b-8403-f6a1d8b6dcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:30.687612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3200e4e-696b-434b-8403-f6a1d8b6dcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:30.766648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f3200e4e-696b-434b-8403-f6a1d8b6dcd1 None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:30.767797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3200e4e-696b-434b-8403-f6a1d8b6dcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:30.872606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 697/1411] 10.222.0.85 () {58 vars in 1015 bytes} [Wed Nov 2 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:30.887321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0116d-efea-4e94-922b-d81105fce654 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:30.887869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0116d-efea-4e94-922b-d81105fce654 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:30.888141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0116d-efea-4e94-922b-d81105fce654 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:30.889496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 715/1412] 10.222.0.85 () {54 vars in 918 bytes} [Wed Nov 2 12:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:30.894509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04bc21-7b2f-439c-8b30-b5d4e35fc3b1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:30.894762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04bc21-7b2f-439c-8b30-b5d4e35fc3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:30.895030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04bc21-7b2f-439c-8b30-b5d4e35fc3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:30.970076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:30.973765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-db04bc21-7b2f-439c-8b30-b5d4e35fc3b1 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:30.975466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-db04bc21-7b2f-439c-8b30-b5d4e35fc3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:31.094657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 698/1413] 10.222.0.85 () {58 vars in 1015 bytes} [Wed Nov 2 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:33.735505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e880838-a8aa-4a0b-9550-27c12f93ff7e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:33.736018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e880838-a8aa-4a0b-9550-27c12f93ff7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:33.736018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e880838-a8aa-4a0b-9550-27c12f93ff7e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:33.737331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:33.743227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbbddb-a231-4cf5-a85f-0f973e243183 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:33.743695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbbddb-a231-4cf5-a85f-0f973e243183 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:33.743794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbbddb-a231-4cf5-a85f-0f973e243183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:33.818676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a3cbbddb-a231-4cf5-a85f-0f973e243183 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:33.820435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3cbbddb-a231-4cf5-a85f-0f973e243183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:33.876910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:33.921564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:34.226615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e880838-a8aa-4a0b-9550-27c12f93ff7e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:34.228196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2e880838-a8aa-4a0b-9550-27c12f93ff7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:34.380696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e880838-a8aa-4a0b-9550-27c12f93ff7e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:34.456527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:34.499632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:34.500235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:34.500507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:34.500773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:34.502404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:34.596186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:34.597824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd1f0f-7113-431a-b4e7-5b30fd78afdc service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:34.617120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 717/1416] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:35.790762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058b3a5-95d3-4ef6-a333-8bf2a9b4bfcd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:35.791301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058b3a5-95d3-4ef6-a333-8bf2a9b4bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:35.791301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058b3a5-95d3-4ef6-a333-8bf2a9b4bfcd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:35.792605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:35.799106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e1622-dfee-4b90-b5db-815c90630ed6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:35.799840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e1622-dfee-4b90-b5db-815c90630ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:35.799840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e1622-dfee-4b90-b5db-815c90630ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:35.876773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c31e1622-dfee-4b90-b5db-815c90630ed6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:35.878039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c31e1622-dfee-4b90-b5db-815c90630ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:35.906104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:35.976334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:36.141917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058b3a5-95d3-4ef6-a333-8bf2a9b4bfcd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:36.143344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a058b3a5-95d3-4ef6-a333-8bf2a9b4bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:36.290139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058b3a5-95d3-4ef6-a333-8bf2a9b4bfcd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:36.330012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:36.405186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:36.405761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:36.405970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:36.406229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:36.407839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:36.500140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:36.501852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9204cdf-536f-416b-a8f1-66db69a40d7a service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:36.520579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 701/1419] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:37.670722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43a0f8-ae8c-44e1-99d8-68204d59c894 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:37.671327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43a0f8-ae8c-44e1-99d8-68204d59c894 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:37.671327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43a0f8-ae8c-44e1-99d8-68204d59c894 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:37.672604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:37.680130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389430e-f39b-45ba-9f7a-6eb493fb49f9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:37.680640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389430e-f39b-45ba-9f7a-6eb493fb49f9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:37.680738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389430e-f39b-45ba-9f7a-6eb493fb49f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:37.739862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:37.763700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8389430e-f39b-45ba-9f7a-6eb493fb49f9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:37.764910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8389430e-f39b-45ba-9f7a-6eb493fb49f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:37.871273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:38.026400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43a0f8-ae8c-44e1-99d8-68204d59c894 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:38.027877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-df43a0f8-ae8c-44e1-99d8-68204d59c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:38.180051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43a0f8-ae8c-44e1-99d8-68204d59c894 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:38.192566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:38.298774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:38.299297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:38.299566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:38.299828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:38.301510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:38.393192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:38.394785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7144d-9eb4-418e-8bbb-8d880a972856 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:38.413230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 720/1422] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:39.536081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415da7e-f69d-47d2-b6c0-d918bef2c7f1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:39.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415da7e-f69d-47d2-b6c0-d918bef2c7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:39.536680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415da7e-f69d-47d2-b6c0-d918bef2c7f1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:39.538088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:39.544739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd071183-615b-42e6-8e47-97c76d5a0a7a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:39.545268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd071183-615b-42e6-8e47-97c76d5a0a7a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:39.545393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd071183-615b-42e6-8e47-97c76d5a0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:39.565258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:39.623753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fd071183-615b-42e6-8e47-97c76d5a0a7a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:39.625280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd071183-615b-42e6-8e47-97c76d5a0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:39.722956 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:39.874292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415da7e-f69d-47d2-b6c0-d918bef2c7f1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:39.875899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5415da7e-f69d-47d2-b6c0-d918bef2c7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:39.992756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:40.030125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415da7e-f69d-47d2-b6c0-d918bef2c7f1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:40.148428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:40.149086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:40.149361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:40.149626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:40.151239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:40.229405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:40.243721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:40.245286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c0b23-cf41-42b4-b157-4c553d285d2b service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:40.264233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 704/1425] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:41.403857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca9b8-bfe0-4e2d-9823-168c0d8423f4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:41.404467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca9b8-bfe0-4e2d-9823-168c0d8423f4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:41.404467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca9b8-bfe0-4e2d-9823-168c0d8423f4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:41.405828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:41.412730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b30b5-72cb-4405-9d70-9a3d1d096268 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:41.413497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b30b5-72cb-4405-9d70-9a3d1d096268 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:41.413497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b30b5-72cb-4405-9d70-9a3d1d096268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:41.492613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-056b30b5-72cb-4405-9d70-9a3d1d096268 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:41.493890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-056b30b5-72cb-4405-9d70-9a3d1d096268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:41.594965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:41.751088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca9b8-bfe0-4e2d-9823-168c0d8423f4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:41.752619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d9ca9b8-bfe0-4e2d-9823-168c0d8423f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:41.840366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:41.902922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca9b8-bfe0-4e2d-9823-168c0d8423f4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:42.016510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:42.017050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:42.017332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:42.017603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:42.019212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:42.067840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:42.118277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:42.119865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf8c9-3f50-444e-94a7-3e81aa71d565 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:42.138907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 723/1428] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:43.207773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878b81-676a-4647-99e8-4f63abd59633 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:43.208362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878b81-676a-4647-99e8-4f63abd59633 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:43.208362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878b81-676a-4647-99e8-4f63abd59633 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:43.209814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:43.217362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a3d4-db35-455c-b1ef-19ca646c26e5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:43.218173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a3d4-db35-455c-b1ef-19ca646c26e5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:43.218173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072a3d4-db35-455c-b1ef-19ca646c26e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:43.295791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3072a3d4-db35-455c-b1ef-19ca646c26e5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:43.297020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3072a3d4-db35-455c-b1ef-19ca646c26e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:43.394868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:43.546233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20878b81-676a-4647-99e8-4f63abd59633 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:43.547784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-20878b81-676a-4647-99e8-4f63abd59633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:43.598967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:43.700836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20878b81-676a-4647-99e8-4f63abd59633 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:43.819449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:43.819989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:43.820262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:43.820529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:43.822177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:43.829424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:43.920197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:43.921850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22557cae-60fc-4eb2-8465-03ab6c318fdb service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:43.941158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 707/1431] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:45.025202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a56ea8-10b0-4964-99c6-cce68629d335 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:45.025781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a56ea8-10b0-4964-99c6-cce68629d335 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:45.025781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a56ea8-10b0-4964-99c6-cce68629d335 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:45.027086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:45.033452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3479734-3142-4b4e-889a-d183f8b1a5da None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:45.033956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3479734-3142-4b4e-889a-d183f8b1a5da None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:45.034056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3479734-3142-4b4e-889a-d183f8b1a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:45.110289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c3479734-3142-4b4e-889a-d183f8b1a5da None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:45.111505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c3479734-3142-4b4e-889a-d183f8b1a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:45.181352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:45.211499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:45.361704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a56ea8-10b0-4964-99c6-cce68629d335 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:45.363222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-08a56ea8-10b0-4964-99c6-cce68629d335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:45.381621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:45.508337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a56ea8-10b0-4964-99c6-cce68629d335 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:45.595669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:45.623064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:45.623666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:45.623939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:45.624196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:45.625839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:45.717546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:45.719146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af3755-d8fb-43c9-a5d3-7072df4d1e06 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:45.738401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 726/1434] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:46.808191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefb0e3-b5a3-450f-a16e-d9390d9052ca None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:46.809027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefb0e3-b5a3-450f-a16e-d9390d9052ca None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:46.809027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefb0e3-b5a3-450f-a16e-d9390d9052ca None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:46.810071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:46.816745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3df65-0b58-40dd-a388-7fead00424da None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:46.817298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3df65-0b58-40dd-a388-7fead00424da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:46.817298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3df65-0b58-40dd-a388-7fead00424da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:46.897250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3ac3df65-0b58-40dd-a388-7fead00424da None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:46.898525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ac3df65-0b58-40dd-a388-7fead00424da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:46.943117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:46.999609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:47.153962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefb0e3-b5a3-450f-a16e-d9390d9052ca None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:47.155401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-eaefb0e3-b5a3-450f-a16e-d9390d9052ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:47.326286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefb0e3-b5a3-450f-a16e-d9390d9052ca None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:47.389872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:47.451093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:47.451698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:47.451956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:47.452215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:47.453846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:47.547205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:47.548818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66dfe7-9e16-46d3-9ca8-500b761a7827 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:47.567775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 710/1437] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:48.636496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2243a2b-38e1-4be3-bc65-665df30f6c5b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:48.637074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2243a2b-38e1-4be3-bc65-665df30f6c5b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:48.637074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2243a2b-38e1-4be3-bc65-665df30f6c5b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:48.638657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:48.645212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f29855-13d9-492e-80b6-3e2b4ccb43ab None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:48.645695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f29855-13d9-492e-80b6-3e2b4ccb43ab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:48.645804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f29855-13d9-492e-80b6-3e2b4ccb43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:48.722702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a8f29855-13d9-492e-80b6-3e2b4ccb43ab None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:48.723831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a8f29855-13d9-492e-80b6-3e2b4ccb43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:48.725888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:48.824323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:48.976282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2243a2b-38e1-4be3-bc65-665df30f6c5b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:48.977752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f2243a2b-38e1-4be3-bc65-665df30f6c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:49.126149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2243a2b-38e1-4be3-bc65-665df30f6c5b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:49.155007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:49.240928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:49.241478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:49.241882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:49.242063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:49.243660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:49.339215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:49.340806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d71f0d-abbc-4297-8c95-e0f98c2b8ae3 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:49.359871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 729/1440] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:51.461875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4591e-268c-4dd2-930b-1d135ab7254b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:51.462485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4591e-268c-4dd2-930b-1d135ab7254b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:51.462485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4591e-268c-4dd2-930b-1d135ab7254b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:51.463777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:51.469754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3620-bf78-46e8-8622-30844740b406 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:51.470551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3620-bf78-46e8-8622-30844740b406 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:51.471543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3620-bf78-46e8-8622-30844740b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:51.522486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:51.551973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca9b3620-bf78-46e8-8622-30844740b406 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:51.552599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca9b3620-bf78-46e8-8622-30844740b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:51.671589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:51.957703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4591e-268c-4dd2-930b-1d135ab7254b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:51.959243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fbc4591e-268c-4dd2-930b-1d135ab7254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:52.071243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4591e-268c-4dd2-930b-1d135ab7254b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:52.107433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:52.154388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:52.154994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:52.155273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:52.155540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:52.157221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:52.283482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:52.284859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb5915-f2a2-40ee-986d-2c5e59de8cb4 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:52.305706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 713/1443] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:54.685648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fc66f-b723-4b35-87bb-9bdeacf39765 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:54.686212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fc66f-b723-4b35-87bb-9bdeacf39765 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:54.686212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fc66f-b723-4b35-87bb-9bdeacf39765 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:54.687531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:54.692985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f0c0d-1014-43b3-99df-150598985d12 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:54.693441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f0c0d-1014-43b3-99df-150598985d12 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:54.693561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f0c0d-1014-43b3-99df-150598985d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:54.764373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-680f0c0d-1014-43b3-99df-150598985d12 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:54.765553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-680f0c0d-1014-43b3-99df-150598985d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:54.779353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:54.864316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:22:55.150308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fc66f-b723-4b35-87bb-9bdeacf39765 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:55.151845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-838fc66f-b723-4b35-87bb-9bdeacf39765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:55.255636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fc66f-b723-4b35-87bb-9bdeacf39765 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:55.271141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:55.341784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:55.342379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:55.342671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:55.342936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:55.344526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:55.457794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:55.459222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f2233-2e19-4778-a7ab-a37a9c3ef1b0 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:55.479040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 732/1446] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:22:58.071397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec503e-1bff-4c23-87ea-2ee67a3b8d58 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:58.071989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec503e-1bff-4c23-87ea-2ee67a3b8d58 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:58.071989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec503e-1bff-4c23-87ea-2ee67a3b8d58 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:58.073348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:22:58.078672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdad9fc-bcac-45f8-8af0-b4b7cd76cbb9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:58.079111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdad9fc-bcac-45f8-8af0-b4b7cd76cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:58.079222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdad9fc-bcac-45f8-8af0-b4b7cd76cbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:58.127472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:58.146261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5fdad9fc-bcac-45f8-8af0-b4b7cd76cbb9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:22:58.147470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5fdad9fc-bcac-45f8-8af0-b4b7cd76cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:58.244012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:22:58.536113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec503e-1bff-4c23-87ea-2ee67a3b8d58 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:22:58.538334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18ec503e-1bff-4c23-87ea-2ee67a3b8d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:58.644174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec503e-1bff-4c23-87ea-2ee67a3b8d58 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:22:58.686425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:22:58.723296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:22:58.723809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:22:58.724087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:22:58.724354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:22:58.726565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:22:58.837049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:22:58.838483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc864f-1e5b-4bbf-ae8f-6e6bbd2cca3a service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:22:58.856794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 716/1449] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:01.181969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cd2d0-9ea5-45b9-a71d-827896b21aa4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:01.182552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cd2d0-9ea5-45b9-a71d-827896b21aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:01.182552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cd2d0-9ea5-45b9-a71d-827896b21aa4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:01.184058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:01.189669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc889d-25a9-468e-b16b-bc7e0285640d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:01.190131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc889d-25a9-468e-b16b-bc7e0285640d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:01.190271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc889d-25a9-468e-b16b-bc7e0285640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:01.272511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-78fc889d-25a9-468e-b16b-bc7e0285640d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:01.273773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-78fc889d-25a9-468e-b16b-bc7e0285640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:01.295141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:01.371826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:01.659531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cd2d0-9ea5-45b9-a71d-827896b21aa4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:01.661113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8d3cd2d0-9ea5-45b9-a71d-827896b21aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:01.767064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cd2d0-9ea5-45b9-a71d-827896b21aa4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:01.788106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:01.842423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:01.843026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:01.843306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:01.843580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:01.845206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:01.955286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:01.956751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070272b-2fef-4196-8b3c-0eb9e657d1b1 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:01.975299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 735/1452] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:04.258401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77adfc1-830e-4ea0-8890-56490dd4b14c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:04.258401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77adfc1-830e-4ea0-8890-56490dd4b14c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:04.259731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77adfc1-830e-4ea0-8890-56490dd4b14c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:04.260307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:04.266570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5b55-acd8-4ceb-8c50-123769a8844b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:04.267033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5b55-acd8-4ceb-8c50-123769a8844b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:04.267138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5b55-acd8-4ceb-8c50-123769a8844b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:04.332241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:04.335090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fe0f5b55-acd8-4ceb-8c50-123769a8844b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:04.337237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe0f5b55-acd8-4ceb-8c50-123769a8844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:04.436462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:04.720941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77adfc1-830e-4ea0-8890-56490dd4b14c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:04.722516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e77adfc1-830e-4ea0-8890-56490dd4b14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:04.829294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77adfc1-830e-4ea0-8890-56490dd4b14c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:04.879501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:04.904112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:04.904714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:04.905012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:04.905291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:04.906888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:05.019056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:05.020674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595f6ac-993a-4f3f-b18e-99fde1f82804 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:05.039057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 719/1455] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:07.259531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a99f3-adf2-4026-a3d8-1d77041a028b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:07.260124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a99f3-adf2-4026-a3d8-1d77041a028b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:07.260124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a99f3-adf2-4026-a3d8-1d77041a028b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:07.261447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:07.267087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008935-fc76-42c6-8c40-5b137f22a099 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:07.267488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008935-fc76-42c6-8c40-5b137f22a099 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:07.267670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008935-fc76-42c6-8c40-5b137f22a099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:07.338733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fb008935-fc76-42c6-8c40-5b137f22a099 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:07.339958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb008935-fc76-42c6-8c40-5b137f22a099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:07.370486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:07.438565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:07.728254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a99f3-adf2-4026-a3d8-1d77041a028b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:07.729840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-388a99f3-adf2-4026-a3d8-1d77041a028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:07.831669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a99f3-adf2-4026-a3d8-1d77041a028b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:07.859889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:07.906455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:07.907065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:07.907336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:07.907601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:07.909207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:08.020788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:08.022800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287b988-1b49-4945-b658-d3ddbdf0a3e6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:08.040949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 738/1458] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:10.262050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc0955-a578-4fe9-a00a-264d4cd0971f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:10.262885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc0955-a578-4fe9-a00a-264d4cd0971f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:10.262885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc0955-a578-4fe9-a00a-264d4cd0971f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:10.264210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:10.269683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355a11e-4738-4e5e-ac19-d6438ce55694 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:10.270154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355a11e-4738-4e5e-ac19-d6438ce55694 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:10.270264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355a11e-4738-4e5e-ac19-d6438ce55694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:10.336884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5355a11e-4738-4e5e-ac19-d6438ce55694 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:10.338128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5355a11e-4738-4e5e-ac19-d6438ce55694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:10.346495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:10.436018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:10.720306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc0955-a578-4fe9-a00a-264d4cd0971f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:10.721932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ddbc0955-a578-4fe9-a00a-264d4cd0971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:10.824787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc0955-a578-4fe9-a00a-264d4cd0971f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:10.881128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:10.900164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:10.900765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:10.901069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:10.901340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:10.902953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:11.014073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:11.015516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba439-ca36-416d-b495-30d81d4bf8c9 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:11.033464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 722/1461] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:13.265407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d6874-6bea-46e5-8986-7d204766a5f0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:13.266014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d6874-6bea-46e5-8986-7d204766a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:13.266014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d6874-6bea-46e5-8986-7d204766a5f0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:13.267323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:13.272852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec1e6c-6879-4b76-8876-66d14044c943 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:13.273343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec1e6c-6879-4b76-8876-66d14044c943 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:13.273525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec1e6c-6879-4b76-8876-66d14044c943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:13.342400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d0ec1e6c-6879-4b76-8876-66d14044c943 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:13.343777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d0ec1e6c-6879-4b76-8876-66d14044c943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:13.384725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:13.444124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:13.734479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d6874-6bea-46e5-8986-7d204766a5f0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:13.736076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-890d6874-6bea-46e5-8986-7d204766a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:13.846153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d6874-6bea-46e5-8986-7d204766a5f0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:13.885493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:13.922707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:13.923308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:13.923580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:13.923847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:13.925479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:14.035912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:14.037953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a218d-f808-4e0b-9b8c-baaf8ad844f0 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:14.056019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 741/1464] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:16.280056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd53cd-8d9e-47f7-a306-20098678d74e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:16.280648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd53cd-8d9e-47f7-a306-20098678d74e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:16.280648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd53cd-8d9e-47f7-a306-20098678d74e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:16.282037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:16.287598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bda58d-29d3-4e86-893d-e8883beb6759 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:16.288047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bda58d-29d3-4e86-893d-e8883beb6759 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:16.288161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bda58d-29d3-4e86-893d-e8883beb6759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:16.356495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c1bda58d-29d3-4e86-893d-e8883beb6759 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:16.357757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c1bda58d-29d3-4e86-893d-e8883beb6759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:16.376617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:16.459891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:16.757043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd53cd-8d9e-47f7-a306-20098678d74e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:16.758578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c8dd53cd-8d9e-47f7-a306-20098678d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:16.864406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd53cd-8d9e-47f7-a306-20098678d74e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:16.938794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:16.946073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:16.946685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:16.946959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:16.947227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:16.948882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:17.062863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:17.064302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90d4e0-fc66-44fa-9348-70172b0f660a service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:17.082589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 725/1467] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:19.288006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc805cb9-038e-44e1-afd4-ac0864cff3cc None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:19.288577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc805cb9-038e-44e1-afd4-ac0864cff3cc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:19.288577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc805cb9-038e-44e1-afd4-ac0864cff3cc None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:19.289919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 12:23:19.295335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50534b-0fb0-43f3-b6cb-4dba04274fb3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:19.295818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50534b-0fb0-43f3-b6cb-4dba04274fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:19.295924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50534b-0fb0-43f3-b6cb-4dba04274fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:19.365687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9f50534b-0fb0-43f3-b6cb-4dba04274fb3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:19.366920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9f50534b-0fb0-43f3-b6cb-4dba04274fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:19.418921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:19.471798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:19.760319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc805cb9-038e-44e1-afd4-ac0864cff3cc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:19.761880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc805cb9-038e-44e1-afd4-ac0864cff3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:19.869221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc805cb9-038e-44e1-afd4-ac0864cff3cc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:19.920669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:19.947948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:19.948562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:19.948820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:19.949106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:19.950719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:20.060674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:20.062054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce33c5-b638-4e28-b765-8cb87fe1ca9d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:20.081063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 744/1470] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:23:29.470432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=22 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:30.078723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=66 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:33.994007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b41813-6522-4912-8dd5-32b4fc324914 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:33.994549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b41813-6522-4912-8dd5-32b4fc324914 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:33.994549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b41813-6522-4912-8dd5-32b4fc324914 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:33.995918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 727/1471] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:34.001363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14e776-0f19-479f-9a55-8539daac7926 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:34.001757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14e776-0f19-479f-9a55-8539daac7926 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:34.001915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14e776-0f19-479f-9a55-8539daac7926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:34.069035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ea14e776-0f19-479f-9a55-8539daac7926 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:34.070248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ea14e776-0f19-479f-9a55-8539daac7926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:34.167453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 745/1472] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:34.464231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a55f1-bd8b-4dda-b0e3-75db17bd25e7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:34.464514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a55f1-bd8b-4dda-b0e3-75db17bd25e7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:34.464785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a55f1-bd8b-4dda-b0e3-75db17bd25e7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:34.466160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 728/1473] 10.222.0.85 () {54 vars in 962 bytes} [Wed Nov 2 12:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:34.471592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb02c-4577-4b6f-a7ce-a46f19549001 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:34.471877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb02c-4577-4b6f-a7ce-a46f19549001 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:34.472153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb02c-4577-4b6f-a7ce-a46f19549001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:34.548776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-812fb02c-4577-4b6f-a7ce-a46f19549001 None None] MFA Rules not processed for user `ce6a8adf15e3498d95442b3ad0ef3864`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:34.550038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-812fb02c-4577-4b6f-a7ce-a46f19549001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:34.571732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:34.629901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 746/1474] 10.222.0.85 () {58 vars in 1059 bytes} [Wed Nov 2 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:34.639086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a55f1-bd8b-4dda-b0e3-75db17bd25e7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:34.640756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0b3a55f1-bd8b-4dda-b0e3-75db17bd25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:34.751150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a55f1-bd8b-4dda-b0e3-75db17bd25e7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:34.822339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:34.822936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:34.823206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:34.823467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:34.825073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:34.878692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:34.935082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:34.936535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b8b8b-e84d-4663-a9aa-d5842fe9e863 service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:34.954789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 729/1475] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:37.219101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f848a-2825-484f-96d8-744d3641582a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:37.219683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f848a-2825-484f-96d8-744d3641582a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:37.219683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f848a-2825-484f-96d8-744d3641582a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:37.221021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 747/1476] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:37.226485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259a728-bf82-4065-8334-8cc26a01e7aa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:37.226894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259a728-bf82-4065-8334-8cc26a01e7aa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:37.227032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259a728-bf82-4065-8334-8cc26a01e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:37.293202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6259a728-bf82-4065-8334-8cc26a01e7aa None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:37.294409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6259a728-bf82-4065-8334-8cc26a01e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:37.392755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 730/1477] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:37.664395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f848a-2825-484f-96d8-744d3641582a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:37.665952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b18f848a-2825-484f-96d8-744d3641582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:37.794615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f848a-2825-484f-96d8-744d3641582a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:37.844367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:37.889093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:37.889703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:37.889979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:37.890250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:37.890878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:37.892607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:23:37.892607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:23:37.892607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:23:37.893771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47fdad-b8c2-45f1-a1f7-842813277a74 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:37.913500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 748/1478] 10.222.0.85 () {56 vars in 1108 bytes} [Wed Nov 2 12:23:37 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:23:40.081116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a521f26-1506-4165-b117-58c871886315 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:40.081649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a521f26-1506-4165-b117-58c871886315 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:40.081649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a521f26-1506-4165-b117-58c871886315 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:40.083574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 731/1479] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:40.088800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68d93-f1ad-46a3-9000-09fc2627dba5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:40.089259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68d93-f1ad-46a3-9000-09fc2627dba5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:40.089416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68d93-f1ad-46a3-9000-09fc2627dba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:40.157014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e0a68d93-f1ad-46a3-9000-09fc2627dba5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:40.158164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e0a68d93-f1ad-46a3-9000-09fc2627dba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:40.252591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 749/1480] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:40.541980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a521f26-1506-4165-b117-58c871886315 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:40.543932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2a521f26-1506-4165-b117-58c871886315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:40.553795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:40.695419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a521f26-1506-4165-b117-58c871886315 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:40.778381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:40.810606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:40.811209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:40.811482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:40.811744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:40.813362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:40.912728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:40.914375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f730dd6-915e-44f7-a85f-24299aeb58d1 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:40.932673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 732/1481] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:43.281456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c9673-f583-470a-b020-581b59f0d8ac None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:43.282111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c9673-f583-470a-b020-581b59f0d8ac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:43.282111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c9673-f583-470a-b020-581b59f0d8ac None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:43.283399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 750/1482] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:43.289174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eacf3-6830-4c87-914e-3ebb64857f69 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:43.289609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eacf3-6830-4c87-914e-3ebb64857f69 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:43.289716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eacf3-6830-4c87-914e-3ebb64857f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:43.356017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-254eacf3-6830-4c87-914e-3ebb64857f69 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:43.357256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-254eacf3-6830-4c87-914e-3ebb64857f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:43.404207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:43.454378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 733/1483] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:43.744324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c9673-f583-470a-b020-581b59f0d8ac None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:43.745887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-586c9673-f583-470a-b020-581b59f0d8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:43.771262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:43.849917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c9673-f583-470a-b020-581b59f0d8ac None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:43.928746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:43.929323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:43.929598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:43.929867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:43.931470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:44.007870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:44.043109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:44.044535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f16a4-66ff-4cbd-a55e-b0749ca178c4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:44.063064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 751/1484] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:46.286471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79388e73-62dc-4d00-a1dc-c3fb1fb19d04 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:46.287044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79388e73-62dc-4d00-a1dc-c3fb1fb19d04 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:46.287044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79388e73-62dc-4d00-a1dc-c3fb1fb19d04 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:46.288366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 734/1485] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:46.293898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f063-24e9-4024-ab5b-cdb5d36dc016 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:46.294344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f063-24e9-4024-ab5b-cdb5d36dc016 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:46.294467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f063-24e9-4024-ab5b-cdb5d36dc016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:46.359974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5332f063-24e9-4024-ab5b-cdb5d36dc016 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:46.361303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5332f063-24e9-4024-ab5b-cdb5d36dc016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:46.458797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 752/1486] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:46.746188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79388e73-62dc-4d00-a1dc-c3fb1fb19d04 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:46.747758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-79388e73-62dc-4d00-a1dc-c3fb1fb19d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:46.851434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79388e73-62dc-4d00-a1dc-c3fb1fb19d04 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:46.928245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:46.929796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:46.930488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:46.931077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:46.931514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:46.933384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:47.043613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:47.045213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fecc1-ec91-43e2-9d94-dbac33ed2844 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:47.063385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 735/1487] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:49.304323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21842974-e5ca-4dd4-b70c-fc5a799c88cb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:49.304861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21842974-e5ca-4dd4-b70c-fc5a799c88cb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:49.304861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21842974-e5ca-4dd4-b70c-fc5a799c88cb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:49.306229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 753/1488] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:49.311422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47886db-85ee-457d-9594-ae5ba2f5fefd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:49.311809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47886db-85ee-457d-9594-ae5ba2f5fefd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:49.311956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47886db-85ee-457d-9594-ae5ba2f5fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:49.378447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c47886db-85ee-457d-9594-ae5ba2f5fefd None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:49.380257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c47886db-85ee-457d-9594-ae5ba2f5fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:49.432414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:49.478397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 736/1489] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:49.772842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21842974-e5ca-4dd4-b70c-fc5a799c88cb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:49.774436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-21842974-e5ca-4dd4-b70c-fc5a799c88cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:49.810372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:49.881767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21842974-e5ca-4dd4-b70c-fc5a799c88cb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:49.956355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:49.956952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:49.957259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:49.957527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:49.959122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:50.041136 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:50.070363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:50.071793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fb9c3-4a56-455b-9da1-1f28c05797ab service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:50.090558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 754/1490] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:23:52.534145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c17f6-d8a4-42d0-bfa7-04a02ecfe8d3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:52.534703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c17f6-d8a4-42d0-bfa7-04a02ecfe8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:52.534703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c17f6-d8a4-42d0-bfa7-04a02ecfe8d3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:52.536064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 737/1491] 10.222.0.85 () {54 vars in 927 bytes} [Wed Nov 2 12:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:23:52.541523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1f10-d8d5-4edf-93cc-0990cd18158e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:52.541928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1f10-d8d5-4edf-93cc-0990cd18158e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:52.542087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f1f10-d8d5-4edf-93cc-0990cd18158e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:52.610057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca2f1f10-d8d5-4edf-93cc-0990cd18158e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:23:52.611292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca2f1f10-d8d5-4edf-93cc-0990cd18158e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:52.709469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 755/1492] 10.222.0.85 () {58 vars in 1024 bytes} [Wed Nov 2 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:23:53.005885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c17f6-d8a4-42d0-bfa7-04a02ecfe8d3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:23:53.007519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-831c17f6-d8a4-42d0-bfa7-04a02ecfe8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:53.111821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c17f6-d8a4-42d0-bfa7-04a02ecfe8d3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:23:53.186624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:23:53.187226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:23:53.187503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:23:53.187772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:23:53.189448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:23:53.199342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:23:53.301094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:23:53.302441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafa67a-5d21-4c1b-a319-41ab2358797e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:23:53.320864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 738/1493] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:24:02.707898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=82 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:24:03.318342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=50 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:16.721444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:16.721978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:16.721978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:16.797954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:25:16.799167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:16.896341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 756/1494] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:25:16.904281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:16.904800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:16.904911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:16.982492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:25:16.983623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:17.079382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 739/1495] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:25:17.084088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:17.085316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:17.096447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4942e04-3fc4-4792-9d56-f1198aeafbad None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:17.158124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:17.193381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:17.193980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:17.194253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:17.194518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:17.195149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:25:17.195445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:17.196442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:17.196442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:25:17.196442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:25:17.197351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:17.208527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 757/1496] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:25:17.218157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:17.227292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7b97-1b9d-45b0-b738-51a46c1a18b0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:17.289330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:17.323255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:17.323879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:17.324155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:17.324424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:17.325084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:17.326867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:17.326867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:25:17.326867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:25:17.328065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:17.384593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 740/1497] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:25:17 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:25:17.389030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:17.390569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:17.424662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ec9bf-c3ce-478c-9f6a-82d70c5ee7a6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:17.520006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:17.520589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:17.520856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:17.521151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:17.521777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:17.522760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:17.522760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:25:17.522760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:25:17.523942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:17.569424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 758/1498] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:25:17 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:25:17.573393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:17.574907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:17.637851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce3c87-a50c-44ee-b45f-90a4c3552105 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:17.722534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:17.734356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:17.735576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:17.735847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:17.736138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:17.736778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:17.737845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:17.737845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:25:17.737845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:25:17.738728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:17.751433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 741/1499] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:25:17.755598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:17.757174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:17.768881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4d9b-b1f9-4c77-9e86-5fe463f579b0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:17.794594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:17.865976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:17.866498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:17.866768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:17.867033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:17.867668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:17.868663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:17.868663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:25:17.868663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:25:17.869566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dc4d0-7765-49f1-9de2-62dd1d04fc39 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:17.881752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 759/1500] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:25:17.886301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:17.887849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:17.922372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5bf8-6199-4319-a3e5-24113ffe2fce admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:18.014876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:18.015529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:18.016342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:18.016615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] PATH_INFO: `/v3/projects/5d5b688c1eca4d329919f8de6c491837/users/7617f57c65a945f59c7e017f7fc7d62a/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:18.039124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5b688c1eca4d329919f8de6c491837, user_id=7617f57c65a945f59c7e017f7fc7d62a, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:18.040150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:18.040150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:25:18.040150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:25:18.042121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175f1ec-88a8-48b7-a0db-57fce4fa1872 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:18.067159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 742/1501] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:25:17 2022] PUT /identity/v3/projects/5d5b688c1eca4d329919f8de6c491837/users/7617f57c65a945f59c7e017f7fc7d62a/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:25:18.078271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acc42-79dd-43c2-a38c-e39e5b5e884e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:18.078556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acc42-79dd-43c2-a38c-e39e5b5e884e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:18.078827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acc42-79dd-43c2-a38c-e39e5b5e884e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:18.137960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-050acc42-79dd-43c2-a38c-e39e5b5e884e None None] MFA Rules not processed for user `7617f57c65a945f59c7e017f7fc7d62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:25:18.139092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-050acc42-79dd-43c2-a38c-e39e5b5e884e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:18.215040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 760/1502] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:25:18.228750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7078643-6e03-4c98-a2b1-68f520ad5577 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:18.229066 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7078643-6e03-4c98-a2b1-68f520ad5577 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:18.229345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7078643-6e03-4c98-a2b1-68f520ad5577 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:18.230699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 743/1503] 10.222.0.85 () {54 vars in 962 bytes} [Wed Nov 2 12:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:25:18.236093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b69495-277a-41a8-932c-c36e90d42785 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:18.236380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b69495-277a-41a8-932c-c36e90d42785 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:18.236652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b69495-277a-41a8-932c-c36e90d42785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:18.268580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:18.314812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-93b69495-277a-41a8-932c-c36e90d42785 None None] MFA Rules not processed for user `b840491b8d05489c8475e1125dbc0b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:25:18.316035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-93b69495-277a-41a8-932c-c36e90d42785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:18.391089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 761/1504] 10.222.0.85 () {58 vars in 1059 bytes} [Wed Nov 2 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:25:18.398828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7078643-6e03-4c98-a2b1-68f520ad5577 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:18.400351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7078643-6e03-4c98-a2b1-68f520ad5577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:18.452127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:18.623469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7078643-6e03-4c98-a2b1-68f520ad5577 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:18.698939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:18.699577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:18.699850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:18.700116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:18.701759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:18.801235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:18.802674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:18.806177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:18.821839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 744/1505] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:25:18.867765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad75b-6647-4fd3-bc3a-291c7d5111cc None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:18.868175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad75b-6647-4fd3-bc3a-291c7d5111cc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:18.868305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad75b-6647-4fd3-bc3a-291c7d5111cc None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:18.869586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 762/1506] 10.222.0.85 () {52 vars in 830 bytes} [Wed Nov 2 12:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:25:18.873172 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:18.874535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:18.978685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ed164-b6bb-41f8-aa98-182d552ee375 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:19.070626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:19.071236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:19.071507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:19.071774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] PATH_INFO: `/v3/users/7617f57c65a945f59c7e017f7fc7d62a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:19.085919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7617f57c65a945f59c7e017f7fc7d62a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:19.086942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:19.086942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:25:19.086942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:25:19.088128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:19.097537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:19.154761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7617f57c65a945f59c7e017f7fc7d62a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:19.163814 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7617f57c65a945f59c7e017f7fc7d62a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:19.164357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7617f57c65a945f59c7e017f7fc7d62a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:19.164805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7617f57c65a945f59c7e017f7fc7d62a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:19.179348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7617f57c65a945f59c7e017f7fc7d62a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:19.208486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 745/1507] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:25:18 2022] DELETE /identity/v3/users/7617f57c65a945f59c7e017f7fc7d62a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:25:19.212832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:19.213150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:19.213429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:19.270977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:25:19.272113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:19.346051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:19.359124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 763/1508] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:25:19.369316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:19.370528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:19.470597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4ae6d-3316-4558-9a71-beaf2b369a00 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:19.526772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:19.548809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:19.549462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:19.549734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:19.549997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:19.551647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:19.661354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:19.662794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7590e-4546-4ab0-a328-9180d431dad9 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:19.680905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 746/1509] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:25:20.054257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:25:20.055529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:25:20.067806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e3867-944f-4653-9a2a-caa7d8949bf2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:25:20.175527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:25:20.176223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:25:20.176506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:25:20.176776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] PATH_INFO: `/v3/projects/5d5b688c1eca4d329919f8de6c491837` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:25:20.187156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5b688c1eca4d329919f8de6c491837)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:25:20.188174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:25:20.188174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:25:20.188174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:25:20.189424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:25:20.247560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5b688c1eca4d329919f8de6c491837', 'request_id': 'req-103f7649-860f-4b04-b307-1a6fd4869947'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:20.260757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5b688c1eca4d329919f8de6c491837', 'request_id': 'req-103f7649-860f-4b04-b307-1a6fd4869947'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:20.269895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5b688c1eca4d329919f8de6c491837', 'request_id': 'req-103f7649-860f-4b04-b307-1a6fd4869947'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:20.294633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] The token cache is being invalidate because project 5d5b688c1eca4d329919f8de6c491837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:25:20.294919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:20.295438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-103f7649-860f-4b04-b307-1a6fd4869947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:25:20.296814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 764/1510] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:25:20 2022] DELETE /identity/v3/projects/5d5b688c1eca4d329919f8de6c491837 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:25:29.678690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=62 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:25:30.292552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:26:50.683141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917147c-f83c-4a3c-9885-8d7e4d72f643 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:50.683732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917147c-f83c-4a3c-9885-8d7e4d72f643 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:50.683732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917147c-f83c-4a3c-9885-8d7e4d72f643 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:50.685076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:26:50.690358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82713-0a17-4788-9621-6b02724feaa0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:50.690858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82713-0a17-4788-9621-6b02724feaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:50.690966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c82713-0a17-4788-9621-6b02724feaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:50.773903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b2c82713-0a17-4788-9621-6b02724feaa0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:26:50.775106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2c82713-0a17-4788-9621-6b02724feaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:26:50.879993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:26:51.032980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917147c-f83c-4a3c-9885-8d7e4d72f643 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:26:51.035774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2917147c-f83c-4a3c-9885-8d7e4d72f643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:26:51.152459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917147c-f83c-4a3c-9885-8d7e4d72f643 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:26:51.158237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176f129-211a-4d9b-8073-a847e9e8cbea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:26:51.158868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176f129-211a-4d9b-8073-a847e9e8cbea service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:51.159785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176f129-211a-4d9b-8073-a847e9e8cbea service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:51.160177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176f129-211a-4d9b-8073-a847e9e8cbea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:51.168563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176f129-211a-4d9b-8073-a847e9e8cbea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:26:51.170164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176f129-211a-4d9b-8073-a847e9e8cbea service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:26:51.179665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 748/1513] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:26:52.274590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1a4cc-8d79-403a-abf0-8e01288a1a48 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:52.275271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1a4cc-8d79-403a-abf0-8e01288a1a48 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:52.275271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1a4cc-8d79-403a-abf0-8e01288a1a48 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:52.276481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:26:52.282674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0c1b2-902e-4e3f-9dd4-1d1658b44258 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:52.283177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0c1b2-902e-4e3f-9dd4-1d1658b44258 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:52.283292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0c1b2-902e-4e3f-9dd4-1d1658b44258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:52.303736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8dd0c1b2-902e-4e3f-9dd4-1d1658b44258 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:26:52.304930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8dd0c1b2-902e-4e3f-9dd4-1d1658b44258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:26:52.313153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:26:52.465943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1a4cc-8d79-403a-abf0-8e01288a1a48 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:26:52.483724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1a4cc-8d79-403a-abf0-8e01288a1a48 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:26:52.489307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f874c6-3d92-4520-8b95-9fc737f740f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:26:52.489995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f874c6-3d92-4520-8b95-9fc737f740f0 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:52.490277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f874c6-3d92-4520-8b95-9fc737f740f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:52.490608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f874c6-3d92-4520-8b95-9fc737f740f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:52.499539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f874c6-3d92-4520-8b95-9fc737f740f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:26:52.501255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f874c6-3d92-4520-8b95-9fc737f740f0 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:26:52.511257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 767/1516] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:26:53.093088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e1c86-2eae-4814-9feb-4e76f3161f94 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:53.093366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e1c86-2eae-4814-9feb-4e76f3161f94 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:53.093633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e1c86-2eae-4814-9feb-4e76f3161f94 None None] PATH_INFO: `/v3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:53.094910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 750/1517] 10.222.0.87 () {54 vars in 925 bytes} [Wed Nov 2 12:26:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:26:53.100869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e376e1-51f6-4837-9683-82d210cedb71 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:53.101174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e376e1-51f6-4837-9683-82d210cedb71 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:53.101447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e376e1-51f6-4837-9683-82d210cedb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:53.161304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-76e376e1-51f6-4837-9683-82d210cedb71 None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:26:53.162516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-76e376e1-51f6-4837-9683-82d210cedb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:26:53.179697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:26:53.234915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 768/1518] 10.222.0.87 () {58 vars in 1015 bytes} [Wed Nov 2 12:26:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:26:56.841436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e75ffd-367d-419d-b203-84ce0e4c551e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:56.841906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e75ffd-367d-419d-b203-84ce0e4c551e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:56.842010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e75ffd-367d-419d-b203-84ce0e4c551e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:56.843292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 751/1519] 10.222.0.85 () {54 vars in 961 bytes} [Wed Nov 2 12:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:26:56.848229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b9229-44d5-4061-8d74-290e96432ac1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:56.848635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b9229-44d5-4061-8d74-290e96432ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:56.848773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b9229-44d5-4061-8d74-290e96432ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:56.869165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-303b9229-44d5-4061-8d74-290e96432ac1 None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:26:56.870358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-303b9229-44d5-4061-8d74-290e96432ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:26:56.879601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 769/1520] 10.222.0.85 () {58 vars in 1058 bytes} [Wed Nov 2 12:26:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:26:56.887716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e75ffd-367d-419d-b203-84ce0e4c551e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:26:56.901667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e75ffd-367d-419d-b203-84ce0e4c551e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:26:56.907650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567f318-e2ba-4cbd-8260-4c0199d264c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:26:56.908244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567f318-e2ba-4cbd-8260-4c0199d264c6 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:26:56.908519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567f318-e2ba-4cbd-8260-4c0199d264c6 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:26:56.908790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567f318-e2ba-4cbd-8260-4c0199d264c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:26:56.917932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567f318-e2ba-4cbd-8260-4c0199d264c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:26:56.919523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567f318-e2ba-4cbd-8260-4c0199d264c6 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:26:56.931810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 752/1521] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:26:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:27:03.973295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491ce0-73d9-465d-b635-d426ecb69b85 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:03.973851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491ce0-73d9-465d-b635-d426ecb69b85 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:03.973851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491ce0-73d9-465d-b635-d426ecb69b85 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:03.975177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:27:03.981518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e981494-f905-4740-ac7e-f91325fa19ad None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:03.981964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e981494-f905-4740-ac7e-f91325fa19ad None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:03.982075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e981494-f905-4740-ac7e-f91325fa19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:04.001868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6e981494-f905-4740-ac7e-f91325fa19ad None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:27:04.003100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6e981494-f905-4740-ac7e-f91325fa19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:04.011254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:27:04.163067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491ce0-73d9-465d-b635-d426ecb69b85 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:27:04.178602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491ce0-73d9-465d-b635-d426ecb69b85 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:27:04.183991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cdba8-8be9-4ccf-ba2b-f2804fa3dfc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:27:04.184578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cdba8-8be9-4ccf-ba2b-f2804fa3dfc0 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:04.184849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cdba8-8be9-4ccf-ba2b-f2804fa3dfc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:04.185145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cdba8-8be9-4ccf-ba2b-f2804fa3dfc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:04.193113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cdba8-8be9-4ccf-ba2b-f2804fa3dfc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:27:04.194673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cdba8-8be9-4ccf-ba2b-f2804fa3dfc0 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:27:04.204428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 771/1524] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:27:05.271631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284cbf-0699-4f93-bbb2-0292c69b0650 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:05.272202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284cbf-0699-4f93-bbb2-0292c69b0650 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:05.272202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284cbf-0699-4f93-bbb2-0292c69b0650 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:05.273615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 12:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:27:05.280018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc7737-42c9-46a8-ae6a-7102a209d98b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:05.280465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc7737-42c9-46a8-ae6a-7102a209d98b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:05.280573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc7737-42c9-46a8-ae6a-7102a209d98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:05.300756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1adc7737-42c9-46a8-ae6a-7102a209d98b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:27:05.302314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1adc7737-42c9-46a8-ae6a-7102a209d98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:05.310570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 12:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:27:05.472657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284cbf-0699-4f93-bbb2-0292c69b0650 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:27:05.487692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284cbf-0699-4f93-bbb2-0292c69b0650 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:27:05.493106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb6b0f-3dc2-4367-8a51-992c48b6e7e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:27:05.493687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb6b0f-3dc2-4367-8a51-992c48b6e7e2 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:05.493963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb6b0f-3dc2-4367-8a51-992c48b6e7e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:05.494236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb6b0f-3dc2-4367-8a51-992c48b6e7e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:05.502265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb6b0f-3dc2-4367-8a51-992c48b6e7e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:27:05.503845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb6b0f-3dc2-4367-8a51-992c48b6e7e2 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:27:05.513564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 755/1527] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:27:09.741542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b20b-ab47-491d-a86d-84e856d7eb67 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:09.742095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b20b-ab47-491d-a86d-84e856d7eb67 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:09.742095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b20b-ab47-491d-a86d-84e856d7eb67 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:09.743357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:27:09.748572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001084cf-913b-424c-89f2-7d5c57b5101d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:09.749067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001084cf-913b-424c-89f2-7d5c57b5101d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:09.749169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001084cf-913b-424c-89f2-7d5c57b5101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:09.769700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-001084cf-913b-424c-89f2-7d5c57b5101d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:27:09.770906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-001084cf-913b-424c-89f2-7d5c57b5101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:09.778889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:27:10.062626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1b20b-ab47-491d-a86d-84e856d7eb67 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:27:10.065325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ada1b20b-ab47-491d-a86d-84e856d7eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:10.148921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1b20b-ab47-491d-a86d-84e856d7eb67 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:27:10.152505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45120adb-cb76-4f3b-af71-ab6ec883a645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:27:10.153166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45120adb-cb76-4f3b-af71-ab6ec883a645 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:10.153448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45120adb-cb76-4f3b-af71-ab6ec883a645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:10.153722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45120adb-cb76-4f3b-af71-ab6ec883a645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:10.162744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45120adb-cb76-4f3b-af71-ab6ec883a645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:27:10.164171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45120adb-cb76-4f3b-af71-ab6ec883a645 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:27:10.177020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 774/1530] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:27:12.337556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431234-86be-44cf-a078-6a2b1b422d74 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:12.338101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431234-86be-44cf-a078-6a2b1b422d74 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:12.338101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431234-86be-44cf-a078-6a2b1b422d74 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:12.339439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:27:12.344769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73c936-d294-4e7c-ab38-07265a55170e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:12.345246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73c936-d294-4e7c-ab38-07265a55170e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:12.345412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73c936-d294-4e7c-ab38-07265a55170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:12.367049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3e73c936-d294-4e7c-ab38-07265a55170e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:27:12.368247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3e73c936-d294-4e7c-ab38-07265a55170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:12.376065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:27:12.662739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431234-86be-44cf-a078-6a2b1b422d74 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:27:12.664518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-89431234-86be-44cf-a078-6a2b1b422d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:12.677743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431234-86be-44cf-a078-6a2b1b422d74 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:27:12.681421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e57c58-e1e1-4437-af55-f7eb4393d242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:27:12.682025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57c58-e1e1-4437-af55-f7eb4393d242 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:12.682315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57c58-e1e1-4437-af55-f7eb4393d242 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:12.682571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57c58-e1e1-4437-af55-f7eb4393d242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:12.692216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e57c58-e1e1-4437-af55-f7eb4393d242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:27:12.693694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e57c58-e1e1-4437-af55-f7eb4393d242 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:27:12.704637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 758/1533] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:27:16.619895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f545-c77f-4968-946c-f2c5afcdacef None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:16.620440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f545-c77f-4968-946c-f2c5afcdacef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:16.620440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f545-c77f-4968-946c-f2c5afcdacef None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:16.621863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:27:16.627978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc298-f26f-4f68-bd54-8297215d709d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:16.628536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc298-f26f-4f68-bd54-8297215d709d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:16.628536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc298-f26f-4f68-bd54-8297215d709d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:16.649770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9e6cc298-f26f-4f68-bd54-8297215d709d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:27:16.650989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9e6cc298-f26f-4f68-bd54-8297215d709d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:16.660315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:27:16.961652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f545-c77f-4968-946c-f2c5afcdacef None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:27:16.963378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7112f545-c77f-4968-946c-f2c5afcdacef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:17.030417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:27:17.045256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f545-c77f-4968-946c-f2c5afcdacef None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:27:17.049626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5ec2-0d98-495d-8ee8-4794770cb543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:27:17.050234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5ec2-0d98-495d-8ee8-4794770cb543 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:17.050542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5ec2-0d98-495d-8ee8-4794770cb543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:17.050815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5ec2-0d98-495d-8ee8-4794770cb543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:17.059980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5ec2-0d98-495d-8ee8-4794770cb543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:27:17.061447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5ec2-0d98-495d-8ee8-4794770cb543 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:27:17.071281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 777/1536] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:27:19.273944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0adcb-2016-4e53-aeb5-55260606fe7c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:19.274563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0adcb-2016-4e53-aeb5-55260606fe7c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:19.274563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0adcb-2016-4e53-aeb5-55260606fe7c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:19.275868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 12:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:27:19.281073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7835aa-9f95-483b-987d-e24ef10bd254 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:19.281488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7835aa-9f95-483b-987d-e24ef10bd254 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:19.281629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7835aa-9f95-483b-987d-e24ef10bd254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:19.301510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7f7835aa-9f95-483b-987d-e24ef10bd254 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:27:19.302712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7f7835aa-9f95-483b-987d-e24ef10bd254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:19.310783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 12:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:27:19.622825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0adcb-2016-4e53-aeb5-55260606fe7c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:27:19.624745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b1f0adcb-2016-4e53-aeb5-55260606fe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:27:19.638218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0adcb-2016-4e53-aeb5-55260606fe7c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:27:19.641965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1bcec-2845-44d3-9010-1df247291caa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:27:19.642592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1bcec-2845-44d3-9010-1df247291caa service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:27:19.642862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1bcec-2845-44d3-9010-1df247291caa service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:27:19.643131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1bcec-2845-44d3-9010-1df247291caa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:27:19.652452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1bcec-2845-44d3-9010-1df247291caa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:27:19.653910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1bcec-2845-44d3-9010-1df247291caa service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:27:19.663781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 761/1539] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:27:29.296034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=11 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:27:29.659766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:52.989954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:52.990936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:52.991092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.038268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:53.041043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:53.065892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 779/1540] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:28:53.077500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.078418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.078418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.135718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:53.138682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:53.165776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 762/1541] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:53.171308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:53.193639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c3435-c0e1-4447-bd85-18dd4af26fa0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:53.210153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:53.211657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.212445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.213261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.214789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:53.216381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:53.219766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:53.252068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 780/1542] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:53.260960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:53.294668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633230c-29be-4b82-a80e-6a71bb6a7379 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:53.308628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:53.309922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.310405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.310884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.312043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:53.315752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:53.416889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 763/1543] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:53 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:28:53.421734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:53.448802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2554f-c0e5-4112-8caf-62ae888105bb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:53.637220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb573989-d794-450f-a4d8-3ddc4bad5781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:53.639105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573989-d794-450f-a4d8-3ddc4bad5781 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.639593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573989-d794-450f-a4d8-3ddc4bad5781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.640044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573989-d794-450f-a4d8-3ddc4bad5781 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.650197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb573989-d794-450f-a4d8-3ddc4bad5781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:53.654186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb573989-d794-450f-a4d8-3ddc4bad5781 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:53.739165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 781/1544] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:53 2022] POST /identity/v3/users => generated 294 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:28:53.745033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:53.772525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1781d9-47cf-42e3-bb63-5762fcff4956 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:53.789081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:53.791012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.791448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.791930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.793223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:53.797435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:53.819231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99c04e-e426-451a-8abc-d8cd9a4583ba None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.819707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99c04e-e426-451a-8abc-d8cd9a4583ba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.820172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99c04e-e426-451a-8abc-d8cd9a4583ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.826059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 764/1545] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:53.831151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:53.885785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae1dd2-23ef-42d0-accf-c6877bd494b2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:53.900936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed99c04e-e426-451a-8abc-d8cd9a4583ba None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:53.903846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593dbad-bca4-46ed-8779-c547e4db1055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:53.904882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593dbad-bca4-46ed-8779-c547e4db1055 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.905402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593dbad-bca4-46ed-8779-c547e4db1055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.905890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593dbad-bca4-46ed-8779-c547e4db1055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.907104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593dbad-bca4-46ed-8779-c547e4db1055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:53.915641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed99c04e-e426-451a-8abc-d8cd9a4583ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:53.919261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593dbad-bca4-46ed-8779-c547e4db1055 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:53.966205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 765/1546] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:28:53.969490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.969991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.970477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:53.975010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 782/1547] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:28:53.979194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30731ee5-f8ba-4d18-9fe9-d37be1a1c88f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:53.979671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30731ee5-f8ba-4d18-9fe9-d37be1a1c88f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:53.980137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30731ee5-f8ba-4d18-9fe9-d37be1a1c88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.055066 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.058796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.075608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-30731ee5-f8ba-4d18-9fe9-d37be1a1c88f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.077774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30731ee5-f8ba-4d18-9fe9-d37be1a1c88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.087912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 766/1548] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.088677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.108022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3815001-fa61-4962-aaf9-a4dd9714fe3b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.116680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 783/1549] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.119107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11485cd-97ab-411d-a7f3-c03512d79076 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.119495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f0ed-e78f-47d9-b9c7-79c00d140320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.119647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11485cd-97ab-411d-a7f3-c03512d79076 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.120174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f0ed-e78f-47d9-b9c7-79c00d140320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.120454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f0ed-e78f-47d9-b9c7-79c00d140320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.120726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f0ed-e78f-47d9-b9c7-79c00d140320 admin admin] PATH_INFO: `/v3/projects/62aa5934d5d14fdb81dd5d4817ab8e74/users/83593f8338c444b7b117eeaf3e4cdcd5/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.126665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11485cd-97ab-411d-a7f3-c03512d79076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.174002 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441f0ed-e78f-47d9-b9c7-79c00d140320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62aa5934d5d14fdb81dd5d4817ab8e74, user_id=83593f8338c444b7b117eeaf3e4cdcd5, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.176988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441f0ed-e78f-47d9-b9c7-79c00d140320 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.199327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c11485cd-97ab-411d-a7f3-c03512d79076 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.201948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c11485cd-97ab-411d-a7f3-c03512d79076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.206707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 767/1550] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:54 2022] PUT /identity/v3/projects/62aa5934d5d14fdb81dd5d4817ab8e74/users/83593f8338c444b7b117eeaf3e4cdcd5/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:54.208670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8d1f-c002-47ba-8cb6-6bd4e62982e2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.209457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8d1f-c002-47ba-8cb6-6bd4e62982e2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.209913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8d1f-c002-47ba-8cb6-6bd4e62982e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.240050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8a7a8d1f-c002-47ba-8cb6-6bd4e62982e2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.242044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8a7a8d1f-c002-47ba-8cb6-6bd4e62982e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.313764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 784/1551] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.313764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca03ce2-bc0f-443e-80c2-22a678e9f0ce None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.313764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca03ce2-bc0f-443e-80c2-22a678e9f0ce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.313764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca03ce2-bc0f-443e-80c2-22a678e9f0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.341249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 768/1552] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.343460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df9968-2c71-4608-a466-0505c4768fd5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.344038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df9968-2c71-4608-a466-0505c4768fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.344581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df9968-2c71-4608-a466-0505c4768fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.358996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0ca03ce2-bc0f-443e-80c2-22a678e9f0ce None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.361345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0ca03ce2-bc0f-443e-80c2-22a678e9f0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.369721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-30df9968-2c71-4608-a466-0505c4768fd5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.371278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-30df9968-2c71-4608-a466-0505c4768fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.376808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 785/1553] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.379579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc207aed-90a8-4f8b-8807-07ebd6f9e0f3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.380047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc207aed-90a8-4f8b-8807-07ebd6f9e0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.380877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc207aed-90a8-4f8b-8807-07ebd6f9e0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.382506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 769/1554] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.384210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.384699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.385184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.412235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.413570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.421696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc207aed-90a8-4f8b-8807-07ebd6f9e0f3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.422419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 770/1555] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.423102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc207aed-90a8-4f8b-8807-07ebd6f9e0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.423296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.435752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 786/1556] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.435752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.435752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.435752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.436543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b9df-fdb0-4f9d-a1fb-decb689ed17f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.443692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.444449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.444872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.445218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.446081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:54.446543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.447821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:28:54.447821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:28:54.447821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:28:54.449065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.458917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.460434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.463450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 771/1557] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:54.464348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.476499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 787/1558] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.477485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.484218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235354d-c96c-4ffa-8047-2224f9891534 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.489481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde8e2-d1e6-4f65-ad57-69aae118bb02 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.494047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.494678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.494957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.495181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.495309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.495838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.495983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:54.496163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.496281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.496376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.497204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:54.497516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.497919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591fbd4-6052-4c02-acb7-81405f04ed95 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.499152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e07165-4498-4eae-84e1-7c2c005925c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.512723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 772/1559] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:54.513213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 788/1560] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:54.519007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4e9a-7860-41cd-8729-afa8cc135bf6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.519488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4e9a-7860-41cd-8729-afa8cc135bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.520036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4e9a-7860-41cd-8729-afa8cc135bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.520201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.521105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.521766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.550935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bf5f4e9a-7860-41cd-8729-afa8cc135bf6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.552213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bf5f4e9a-7860-41cd-8729-afa8cc135bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.552895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.554138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.567481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 773/1561] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.567789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 789/1562] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.567789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.568112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.568112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.568112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.591632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:54.595051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.596343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.607022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 774/1563] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.608224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.619565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faeaf8-e42c-4c59-ac49-af9ed48203de None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.624882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c684564-cd63-4419-a518-21967527ed67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.625751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c684564-cd63-4419-a518-21967527ed67 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.626202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c684564-cd63-4419-a518-21967527ed67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.626780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c684564-cd63-4419-a518-21967527ed67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.627937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c684564-cd63-4419-a518-21967527ed67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.630545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c684564-cd63-4419-a518-21967527ed67 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.638758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78945e5-3a39-4430-882c-64ace4d5074c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.642875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.643499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.643773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.644042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.651881 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:54.662954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.664943 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.667484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 775/1564] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:54.668850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.669173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.669457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.681827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 790/1565] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:28:54.682692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.690878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.692150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.694413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe959854-7fb2-4fd4-986e-03ad0db50b87 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.784313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 776/1566] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.786506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.801582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138f20a-92cd-4ffc-8814-3dd32cf1fcb5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.803205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.804756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.805428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.805962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.806651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:54.807281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.807465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.807631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.807792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.808438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:54.809495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:28:54.809495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:28:54.809495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:28:54.810644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.812079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54c1d-8d57-4beb-8f1f-2f2ad9f41476 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:54.850786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 777/1567] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:54.851303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:54.872011 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 791/1568] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/projects => generated 405 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:54.873730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:54.874045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:54.874339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:54.899478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:54.900743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:54.966467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0c0a4-dc7b-4399-a2b7-6c7d4e162864 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:54.988287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 792/1569] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:54.989936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.004075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303c686-af4c-40a8-b46d-09c29edeca29 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.009894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.010599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.010822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.011097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.011772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:55.012077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.014384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.028848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 793/1570] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:55.029858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.041430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd87ff-b5a6-438c-b10e-ae6decc466a0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.047803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.048430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.049154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.049154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.049675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.051632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.064396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.067116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.067116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.067116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.068062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:55.068664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.071132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.090010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 794/1571] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/users => generated 290 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.093247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 778/1572] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:55.094337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.094518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.114040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65399b80-28e9-432e-81b4-b883c4cba467 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.119945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094516-e996-4aa3-98f6-c659c4e4643b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.122950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.123822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.124242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.124577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.125497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.126453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.127231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:28:55.127231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:28:55.127231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:28:55.127712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.127712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.128102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.128956 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:55.129200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.129448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.131320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.147506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 795/1573] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:55.149564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.156214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:55.164563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266c162-7ade-46e1-9f0c-e6f9fddc1c8d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.166222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:55.175832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.176528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.176901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.177218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.178259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.180673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.185274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 779/1574] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.187184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.210296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130a381-16de-4e19-8784-169710483c05 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.213550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 796/1575] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/users => generated 303 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.214717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.226633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f66fb-5e58-486a-b117-7df89f7539b2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.226961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.227852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.228492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.229148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.230267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:28:55.230890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.232978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.233824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.234176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.234521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.234871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.235387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.237469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.284150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 797/1576] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/projects => generated 393 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.285023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.289278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 780/1577] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:28:55.290869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.332891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089160-eb1c-437f-a43b-42474ecc7649 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.461866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d6b12-4d2f-4593-ae82-63e40e831764 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.476629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.477715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.478191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.478718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.481025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.486929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.487620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.489527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.490434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.492202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.493842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.498181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.512964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 798/1578] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:55.516646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.539360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118193-0015-4ff5-b933-9832f78345fc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.549464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.550500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.550987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.551474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.552673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.555036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 781/1579] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/projects => generated 367 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.556207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.557154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.614153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 799/1580] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.615775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.686930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e008cf2-407a-4f0e-bb2f-58e9ccf584a5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.707844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8792dc-9467-4021-96c5-63346b093e86 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.719807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.720659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.721609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.721609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.722685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.724670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.738792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 800/1581] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:55.739860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.761275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad4ec9-9d4f-434c-a307-7590d036c080 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.769951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.771038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.771491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.771973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.773081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.777769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.797220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.798285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.798786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.799383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.800578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.803922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.824657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 782/1582] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:55.826257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.831170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 801/1583] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:55.835389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.847438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944c123-5565-4524-b5bb-b4fdeb635119 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.865709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:55.869248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:55.957228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd717af3-ad8b-42fa-b1d3-caa198f363a8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.962164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.963122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.963357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.963823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.964034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.964199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.964386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.964787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.965037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.966865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:55.967245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.970781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:55.979472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 802/1584] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:55.980150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:55.992210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fb324-a82a-4754-8e93-8eb3a2211289 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:55.997874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:55.998348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:55.998796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:55.999077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:55.999735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.001745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.013506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 783/1585] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:56.015171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.031893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 803/1586] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:55 2022] POST /identity/v3/users => generated 284 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:56.033321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.046232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba2e59-fc64-4272-ade9-8be084b84f53 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.051574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce189e89-7a7a-4f80-8fbc-dd65af2df781 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.052775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.053457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.053738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.054008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.054703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.056494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.066367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.068320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.068923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.070325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.071271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 804/1587] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.072190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.072492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.078247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.089591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e7fc-cafe-40b1-a143-da812fe11236 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.095616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.096712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.096712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.097092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.097865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.099613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.114361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 805/1588] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.115639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.127633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562bbc-b1b8-451e-ba32-998c80c84d0e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.132183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 784/1589] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:56 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:56.135530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.137134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.138725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.139181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.139564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.140324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.142915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.162351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da147e6-3344-4bb4-a736-851c91e6ad3d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.171939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.173184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.173608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 806/1590] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:56 2022] POST /identity/v3/users => generated 290 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:56.174841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.175046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] PATH_INFO: `/v3/projects/565eec3408344c2e84f2988fb2230c46/users/8285583445bc4964a3b4b66b462d9212/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.177941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.191396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbbdf1-b6c2-4c8c-8f35-266856f3d93a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.204404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.205745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.206236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.206721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.212471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.216763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565eec3408344c2e84f2988fb2230c46, user_id=8285583445bc4964a3b4b66b462d9212, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.220043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.222905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.239950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 807/1591] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.240604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.263450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 785/1592] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:56 2022] PUT /identity/v3/projects/565eec3408344c2e84f2988fb2230c46/users/8285583445bc4964a3b4b66b462d9212/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:56.263450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.280832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f00404-3e71-46c6-98c2-e33c7468d5d8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.362598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.363700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.364167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.364594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.365602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.367937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.383702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 808/1593] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.384970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.411563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfc07a-0843-4d62-92ea-4343a6666344 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.413406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bf73c-ce28-4059-b3fe-ce2b7327c977 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.423598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.425396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.426047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.426547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.426681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.427030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.427523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] PATH_INFO: `/v3/projects/88ea7c290e7b4470b7b336ec9da930ca/users/4bbf9a39008440509a5d2676bab1d28b/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.428011 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] PATH_INFO: `/v3/projects/2c34199e1871487882981bb6a96a6e15/users/beacd62576aa4df49b561bef591e665f/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.458600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ea7c290e7b4470b7b336ec9da930ca, user_id=4bbf9a39008440509a5d2676bab1d28b, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.464587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.471965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c34199e1871487882981bb6a96a6e15, user_id=beacd62576aa4df49b561bef591e665f, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.474252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:28:56.474252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:28:56.474252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:28:56.477748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.491857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 786/1594] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:56 2022] PUT /identity/v3/projects/88ea7c290e7b4470b7b336ec9da930ca/users/4bbf9a39008440509a5d2676bab1d28b/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:56.508451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.541941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 809/1595] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:56 2022] PUT /identity/v3/projects/2c34199e1871487882981bb6a96a6e15/users/beacd62576aa4df49b561bef591e665f/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:56.541941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.566392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:56.583010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:56.630794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3399ef0-eff2-442b-a51d-81c6c998b960 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.664226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50547ef4-dc66-48b0-ae93-d75c05c8c7ea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.670415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.671060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.671344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.671598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.672279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.674264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.697754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 810/1596] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.698852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.712229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd95f-1276-4a30-a4c1-1c54bdf0de30 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.719476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ade4-8022-4f79-912a-04272f9e4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.720159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ade4-8022-4f79-912a-04272f9e4c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.720433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ade4-8022-4f79-912a-04272f9e4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.720699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ade4-8022-4f79-912a-04272f9e4c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.721578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ade4-8022-4f79-912a-04272f9e4c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.723248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ade4-8022-4f79-912a-04272f9e4c4b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.741080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 811/1597] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.741080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393cd8-c565-49b5-85b6-2778c5076759 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.741080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393cd8-c565-49b5-85b6-2778c5076759 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.741080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393cd8-c565-49b5-85b6-2778c5076759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.773545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-55393cd8-c565-49b5-85b6-2778c5076759 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:56.776345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.777126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.777379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.777656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.778796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-55393cd8-c565-49b5-85b6-2778c5076759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:56.779361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.781281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.796733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 787/1598] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.804715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 812/1599] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:56.805957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.806140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.806140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.806140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.824553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c3269-c7f2-4ef0-bcaf-0ea05b02eb67 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.838707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4eddf-a221-48bb-a29c-349904edfcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.839870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4eddf-a221-48bb-a29c-349904edfcb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.841497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4eddf-a221-48bb-a29c-349904edfcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.843476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4eddf-a221-48bb-a29c-349904edfcb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.844542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4eddf-a221-48bb-a29c-349904edfcb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.848126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4eddf-a221-48bb-a29c-349904edfcb8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.852465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] MFA Rules not processed for user `4bbf9a39008440509a5d2676bab1d28b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:56.854203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:56.872512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 788/1600] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.874995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.875596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.876411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.935633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] MFA Rules not processed for user `beacd62576aa4df49b561bef591e665f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:56.938961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:56.949766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 813/1601] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:56.951843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:56.968651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8fe77-4c11-41c9-8d94-82077ff5c2d6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:56.975105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:56.978735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:56.978735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:56.978735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:56.978735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:56.979650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:56.998749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 814/1602] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:56.999766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.014543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d226-6f74-4ec6-86d9-ad5368f63566 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.025226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.026082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.026438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.026785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] PATH_INFO: `/v3/projects/3dfb62f918dd41aea2e6d49286476dd0/users/7c5ccd8c99934f60b4ca26c4990b7412/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.047604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfb62f918dd41aea2e6d49286476dd0, user_id=7c5ccd8c99934f60b4ca26c4990b7412, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.050796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.070004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 815/1603] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:56 2022] PUT /identity/v3/projects/3dfb62f918dd41aea2e6d49286476dd0/users/7c5ccd8c99934f60b4ca26c4990b7412/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:57.071191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.103281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 789/1604] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:57.103281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.148025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:57.193960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e2346-b633-4507-9f69-ecfcb1ba75a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.202189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e3be7-41fe-4463-8021-32526e7b5041 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.209468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.210906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.211801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.212515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.214920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.220986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.243425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 816/1605] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:57.243425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.246567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:57.259730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce09dc-ef79-4b78-ac58-55f407b70471 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.271218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.272738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.273230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.273691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] PATH_INFO: `/v3/projects/36d6c3038bdf4479836310d68284de32/users/092baf9aa02e48b08fdb2b2685ddaae2/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.303546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d6c3038bdf4479836310d68284de32, user_id=092baf9aa02e48b08fdb2b2685ddaae2, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.305644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.306387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.306587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.306840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.307427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.328986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.333168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 817/1606] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:57 2022] PUT /identity/v3/projects/36d6c3038bdf4479836310d68284de32/users/092baf9aa02e48b08fdb2b2685ddaae2/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:57.334082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.334696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.363031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 790/1607] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:28:57.364949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.384554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5be437-96f6-4d49-a89d-7dbf8a73ede4 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.531366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa13e72-101e-443b-8f24-69861b34f0a8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.569024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80bfdc-5232-461f-8047-9b64f2ddd064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.570208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80bfdc-5232-461f-8047-9b64f2ddd064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.570667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80bfdc-5232-461f-8047-9b64f2ddd064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.571209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80bfdc-5232-461f-8047-9b64f2ddd064 admin admin] PATH_INFO: `/v3/projects/5abb17741f2241d7b15d2c1b48b20f2b/users/499f80abd04b4580b0abbab52ce303e1/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.578493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf711c-fee4-4d5d-80ed-d6aad212201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.580332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf711c-fee4-4d5d-80ed-d6aad212201f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.580819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf711c-fee4-4d5d-80ed-d6aad212201f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.581122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf711c-fee4-4d5d-80ed-d6aad212201f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.583408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf711c-fee4-4d5d-80ed-d6aad212201f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.592866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf711c-fee4-4d5d-80ed-d6aad212201f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.594886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80bfdc-5232-461f-8047-9b64f2ddd064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abb17741f2241d7b15d2c1b48b20f2b, user_id=499f80abd04b4580b0abbab52ce303e1, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.599807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80bfdc-5232-461f-8047-9b64f2ddd064 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.634018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 791/1608] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:57 2022] PUT /identity/v3/projects/5abb17741f2241d7b15d2c1b48b20f2b/users/499f80abd04b4580b0abbab52ce303e1/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:57.634018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882220b-5e1a-48a7-b264-aab68a15dab3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.634018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882220b-5e1a-48a7-b264-aab68a15dab3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.634018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882220b-5e1a-48a7-b264-aab68a15dab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.656857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d882220b-5e1a-48a7-b264-aab68a15dab3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:57.658189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d882220b-5e1a-48a7-b264-aab68a15dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:57.661272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 818/1609] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:57 2022] POST /identity/v3/projects => generated 403 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:57.665302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58ffb-e9d3-4e67-b2dd-d5e7f45a50ad None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.665627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58ffb-e9d3-4e67-b2dd-d5e7f45a50ad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.665911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58ffb-e9d3-4e67-b2dd-d5e7f45a50ad None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.670094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 819/1610] 10.222.0.85 () {54 vars in 960 bytes} [Wed Nov 2 12:28:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:28:57.671500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58ffb-e9d3-4e67-b2dd-d5e7f45a50ad None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:57.770964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 792/1611] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:57.772937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.773242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.773522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.785583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58ffb-e9d3-4e67-b2dd-d5e7f45a50ad None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:57.792915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ad78-db4a-41cb-9100-9773792c1d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:57.793997 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ad78-db4a-41cb-9100-9773792c1d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.794482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ad78-db4a-41cb-9100-9773792c1d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.794931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ad78-db4a-41cb-9100-9773792c1d32 admin admin] PATH_INFO: `/v3/projects/4052f9d3c4f340eba358014a59e54a11/users/338cd3b7a7b24bbc83d448e508bfaef6/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.821395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ad78-db4a-41cb-9100-9773792c1d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4052f9d3c4f340eba358014a59e54a11, user_id=338cd3b7a7b24bbc83d448e508bfaef6, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:57.823439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] MFA Rules not processed for user `092baf9aa02e48b08fdb2b2685ddaae2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:57.824688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ad78-db4a-41cb-9100-9773792c1d32 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:57.825618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:57.861622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 820/1612] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:57 2022] PUT /identity/v3/projects/4052f9d3c4f340eba358014a59e54a11/users/338cd3b7a7b24bbc83d448e508bfaef6/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:57.867794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022b66-538e-4ef8-ba91-cb9e20f6b304 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:57.868331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022b66-538e-4ef8-ba91-cb9e20f6b304 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:57.868757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022b66-538e-4ef8-ba91-cb9e20f6b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:57.938760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eb022b66-538e-4ef8-ba91-cb9e20f6b304 None None] MFA Rules not processed for user `499f80abd04b4580b0abbab52ce303e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:57.939479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eb022b66-538e-4ef8-ba91-cb9e20f6b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:57.984308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:57.995617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:58.059961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 793/1613] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:58.065976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.093331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc98ab-3058-4675-93dc-e1d7c3b2c76a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.102207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 821/1614] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:58.103698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5c708-a9b1-418e-b9ba-64fad19f5037 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.104087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5c708-a9b1-418e-b9ba-64fad19f5037 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.104426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5c708-a9b1-418e-b9ba-64fad19f5037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.160983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-76d5c708-a9b1-418e-b9ba-64fad19f5037 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:58.163065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-76d5c708-a9b1-418e-b9ba-64fad19f5037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:58.229322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.231440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.231853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.232261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.233364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.236450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:58.273807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 794/1615] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:58 2022] POST /identity/v3/users => generated 308 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:58.273807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.284673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 822/1616] 10.222.0.85 () {58 vars in 1057 bytes} [Wed Nov 2 12:28:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:58.286711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.286912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.287185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.328830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] MFA Rules not processed for user `338cd3b7a7b24bbc83d448e508bfaef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:58.331026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:58.362586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3e769-210f-46bc-a01c-53de6ea98f5d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.367103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.368554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.368934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.369344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.381491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.384557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:58.398866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 795/1617] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:28:58.400114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.452831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 823/1618] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:28:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:58.452831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.460867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7edd9-bd5e-441f-82e0-ffa1cc14d95a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.474331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.476182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.477262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.477907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.479675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.483196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:58.491387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697338be-0a5c-498e-b95f-d3a76e5bd09e service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.503094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.503094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.503622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.504563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.522477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:58.534174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 824/1619] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:28:58 2022] POST /identity/v3/projects => generated 381 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:58.535597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.542956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:58.624847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.627112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:58.642718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 796/1620] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:28:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:28:58.644007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.658769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e08cc2-f3ed-4131-beab-6899a5fa5d28 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.660512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a368f-955c-4b99-9f69-26c554018028 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.678841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8502d-219a-45e6-bfec-c363cf997d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.679769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8502d-219a-45e6-bfec-c363cf997d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.680148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8502d-219a-45e6-bfec-c363cf997d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.680510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8502d-219a-45e6-bfec-c363cf997d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.681524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8502d-219a-45e6-bfec-c363cf997d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.683838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8502d-219a-45e6-bfec-c363cf997d4b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:58.699438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 825/1621] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:58.706306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbecb17-0d24-4fb3-b158-20c08789fefb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.706829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbecb17-0d24-4fb3-b158-20c08789fefb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.707409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbecb17-0d24-4fb3-b158-20c08789fefb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.709065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 826/1622] 10.222.0.85 () {54 vars in 960 bytes} [Wed Nov 2 12:28:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:28:58.716870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbecb17-0d24-4fb3-b158-20c08789fefb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.732586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbecb17-0d24-4fb3-b158-20c08789fefb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.738488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.739561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.740163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.740548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.741496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.744698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:58.795109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 827/1623] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:28:58 2022] POST /identity/v3/users => generated 297 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:28:58.795109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:58.844565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227ea2c-43fd-4ecf-ba6d-1c6be3bd09c0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:58.865963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea11a0-2fa9-47fe-885d-531f4c198d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.869689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea11a0-2fa9-47fe-885d-531f4c198d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.869689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea11a0-2fa9-47fe-885d-531f4c198d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.870144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:58.871530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:58.872378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea11a0-2fa9-47fe-885d-531f4c198d9b admin admin] PATH_INFO: `/v3/projects/d9bf83e021ba49f5ac0b10b6d6157507/users/5dc4e186c01142fba4fb6bd55d43e060/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.873561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:58.875541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:58.940872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea11a0-2fa9-47fe-885d-531f4c198d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bf83e021ba49f5ac0b10b6d6157507, user_id=5dc4e186c01142fba4fb6bd55d43e060, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:58.956037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea11a0-2fa9-47fe-885d-531f4c198d9b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:59.009732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 828/1624] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:58 2022] PUT /identity/v3/projects/d9bf83e021ba49f5ac0b10b6d6157507/users/5dc4e186c01142fba4fb6bd55d43e060/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:59.015132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:59.015803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:59.016390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:59.095434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:28:59.097076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:59.101284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:59.111130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:28:59.141914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 797/1625] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:28:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:28:59.159543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:59.213713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346adf80-199c-4102-a14d-28c426a2ccdc service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:59.372213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 829/1626] 10.222.0.85 () {58 vars in 1057 bytes} [Wed Nov 2 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:28:59.373121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:59.419633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:59.421275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:59.421737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:59.422280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:59.423492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:59.427001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:59.436352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:59.482102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 798/1627] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:59.483795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:59.488883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:28:59.520462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143340d0-b6fd-4dbb-90ba-020a1ff55f26 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:59.532854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:59.533918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:59.534332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:59.534809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:59.538719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d5e48-3a0c-4287-8e08-10b3e24566e1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:59.553851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:59.554592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:59.554893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:59.556330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:59.557246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:59.569577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:59.621601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 830/1628] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:28:59.628271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:59.716421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622767-49a9-428d-82ca-5858c3bd9b25 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:28:59.751277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:28:59.752459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:28:59.752934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:28:59.753495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] PATH_INFO: `/v3/projects/cca31d41c4894131b2cdd269c6dc26c8/users/85f33f8322294446a683ef44edb8c631/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:28:59.811938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca31d41c4894131b2cdd269c6dc26c8, user_id=85f33f8322294446a683ef44edb8c631, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:59.827041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:59.855631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:28:59.860037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:28:59.891708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 831/1629] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:59 2022] PUT /identity/v3/projects/cca31d41c4894131b2cdd269c6dc26c8/users/85f33f8322294446a683ef44edb8c631/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:28:59.891708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:59.935663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 799/1630] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:28:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:28:59.939176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:28:59.986823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf525b-5b70-46a6-b43c-70d3420a5537 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:00.164109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e70c4e-abeb-4bb0-b585-edf2b9ae1278 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:00.175373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bac0f-5185-432e-abef-ca2af6955f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:00.176810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bac0f-5185-432e-abef-ca2af6955f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.177410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bac0f-5185-432e-abef-ca2af6955f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.177961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bac0f-5185-432e-abef-ca2af6955f5d admin admin] PATH_INFO: `/v3/projects/d9bf83e021ba49f5ac0b10b6d6157507/users/5dc4e186c01142fba4fb6bd55d43e060/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.182291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bac0f-5185-432e-abef-ca2af6955f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bf83e021ba49f5ac0b10b6d6157507, user_id=5dc4e186c01142fba4fb6bd55d43e060, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:00.188392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bac0f-5185-432e-abef-ca2af6955f5d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:00.216279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 832/1631] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:28:59 2022] PUT /identity/v3/projects/d9bf83e021ba49f5ac0b10b6d6157507/users/5dc4e186c01142fba4fb6bd55d43e060/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:00.219942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb6da5-1a1f-47a8-8fe7-ca5845edb3ea None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.220699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb6da5-1a1f-47a8-8fe7-ca5845edb3ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.221295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb6da5-1a1f-47a8-8fe7-ca5845edb3ea None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.224595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 833/1632] 10.222.0.85 () {54 vars in 912 bytes} [Wed Nov 2 12:29:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:00.233332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.233989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.234458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7d489-ee71-41e1-bd6a-32e9c1fc6b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:00.235177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.235594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7d489-ee71-41e1-bd6a-32e9c1fc6b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.236099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7d489-ee71-41e1-bd6a-32e9c1fc6b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.236625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7d489-ee71-41e1-bd6a-32e9c1fc6b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.240149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7d489-ee71-41e1-bd6a-32e9c1fc6b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:00.242279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7d489-ee71-41e1-bd6a-32e9c1fc6b29 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:00.287553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:00.290429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:00.299456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 800/1633] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:00.301853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48fdb-e1bc-4231-837d-d8062eb2ffa2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.302357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48fdb-e1bc-4231-837d-d8062eb2ffa2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.302813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48fdb-e1bc-4231-837d-d8062eb2ffa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.420489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-afa48fdb-e1bc-4231-837d-d8062eb2ffa2 None None] MFA Rules not processed for user `5dc4e186c01142fba4fb6bd55d43e060`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:00.422701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-afa48fdb-e1bc-4231-837d-d8062eb2ffa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:00.601855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 834/1634] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:00.604631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:00.627071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 801/1635] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:00.628651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61443c3-a18a-400e-8b9d-7cfb59566e81 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.628965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61443c3-a18a-400e-8b9d-7cfb59566e81 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.629254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61443c3-a18a-400e-8b9d-7cfb59566e81 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.637653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 802/1636] 10.222.0.85 () {54 vars in 912 bytes} [Wed Nov 2 12:29:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:00.639026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61443c3-a18a-400e-8b9d-7cfb59566e81 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:00.748731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:00.796466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:00.840648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66a22d-1712-40d8-b88e-9704bb4a8438 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:00.850704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:00.852007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.852739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.853458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] PATH_INFO: `/v3/projects/cca31d41c4894131b2cdd269c6dc26c8/users/85f33f8322294446a683ef44edb8c631/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.858100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca31d41c4894131b2cdd269c6dc26c8, user_id=85f33f8322294446a683ef44edb8c631, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:00.871087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:00.872099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61443c3-a18a-400e-8b9d-7cfb59566e81 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:00.881148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c0701-c314-4954-8d92-06007fed1377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:00.882316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c0701-c314-4954-8d92-06007fed1377 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.882749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c0701-c314-4954-8d92-06007fed1377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.883125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c0701-c314-4954-8d92-06007fed1377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.910213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c0701-c314-4954-8d92-06007fed1377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:00.911755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c0701-c314-4954-8d92-06007fed1377 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:00.960209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 803/1638] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:00.960209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:00.960209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:00.960209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:00.961654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 835/1638] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:00 2022] PUT /identity/v3/projects/cca31d41c4894131b2cdd269c6dc26c8/users/85f33f8322294446a683ef44edb8c631/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:00.963703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:01.038049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:01.044042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:01.142644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb718-8a49-4a03-a2ee-b746047e0cbc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:01.164603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e55c1-1583-424a-a951-4a48237bfe8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:01.165739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e55c1-1583-424a-a951-4a48237bfe8f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:01.166212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e55c1-1583-424a-a951-4a48237bfe8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:01.166702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e55c1-1583-424a-a951-4a48237bfe8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:01.293496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e55c1-1583-424a-a951-4a48237bfe8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:01.295199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e55c1-1583-424a-a951-4a48237bfe8f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:01.308303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 836/1639] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:00 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:01.309714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:01.310093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:01.310377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:01.347958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] MFA Rules not processed for user `85f33f8322294446a683ef44edb8c631`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:01.349282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:01.371510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 804/1640] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:01.479979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 837/1641] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:01.626258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:01.676490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf6d4c-0e5d-4037-8744-1d0504bb5ac7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:01.694453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:01.695659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:01.696477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:01.697268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:01.741231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:01.748127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:01.804719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 805/1642] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:01.935979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:01.951407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ee31f-753a-498e-afd1-5e675e837cc2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:01.966660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:01.967306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:01.967586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:01.967858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:01.995069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:01.999213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:02.017532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:02.040180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 838/1643] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:02.788492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:02.813671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac70ee3-70ae-42d5-8bf6-be6e4c5881ef service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:02.830890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ba3e0-55d9-4bdb-bae2-d489df30a5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:02.831836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ba3e0-55d9-4bdb-bae2-d489df30a5fc service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:02.832355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ba3e0-55d9-4bdb-bae2-d489df30a5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:02.832686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ba3e0-55d9-4bdb-bae2-d489df30a5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:02.874419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:03.083776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ba3e0-55d9-4bdb-bae2-d489df30a5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:03.085699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ba3e0-55d9-4bdb-bae2-d489df30a5fc service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:03.098448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 806/1644] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:03.168262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fe719-2af2-426d-bfbf-f8f7785ad484 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:03.198711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:03.202517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:03.204330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:03.204755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] PATH_INFO: `/v3/users/4bbf9a39008440509a5d2676bab1d28b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:03.210574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bbf9a39008440509a5d2676bab1d28b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:03.212634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:29:03.212634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Nov 02 12:29:03.212634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: Nov 02 12:29:03.215825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:03.413110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bbf9a39008440509a5d2676bab1d28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:03.433417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bbf9a39008440509a5d2676bab1d28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:03.437484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bbf9a39008440509a5d2676bab1d28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:03.439552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bbf9a39008440509a5d2676bab1d28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:03.503261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bbf9a39008440509a5d2676bab1d28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:03.701547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 839/1645] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:02 2022] DELETE /identity/v3/users/4bbf9a39008440509a5d2676bab1d28b => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:29:03.707944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:03.708830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:03.709333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:03.817198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:03.821777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:04.006073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:04.157466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 807/1646] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:29:04.173797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:04.177670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:04.519193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955f3ba-cee3-44e6-882a-0432558f73fc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:04.546548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e5374-e007-4ff6-b654-3405560d2e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:04.547570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5374-e007-4ff6-b654-3405560d2e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:04.548035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5374-e007-4ff6-b654-3405560d2e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:04.548512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5374-e007-4ff6-b654-3405560d2e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:04.626444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e5374-e007-4ff6-b654-3405560d2e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:04.630910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e5374-e007-4ff6-b654-3405560d2e58 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:04.668876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 840/1647] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:05.612398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:05.640298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:05.740110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfdcea-890a-4c6c-a279-5c9e1d10cad0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:05.854229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:05.855329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:05.861921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:05.862649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] PATH_INFO: `/v3/users/beacd62576aa4df49b561bef591e665f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:05.872692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beacd62576aa4df49b561bef591e665f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:05.913264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:06.192815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beacd62576aa4df49b561bef591e665f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:06.216610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beacd62576aa4df49b561bef591e665f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:06.218923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beacd62576aa4df49b561bef591e665f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:06.222730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beacd62576aa4df49b561bef591e665f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:06.260297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beacd62576aa4df49b561bef591e665f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:06.382592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 808/1648] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:05 2022] DELETE /identity/v3/users/beacd62576aa4df49b561bef591e665f => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:06.388131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:06.388572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:06.388694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:06.483264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:06.491930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:06.597630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:06.894473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 841/1649] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:06.915501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:06.917133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:07.324861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a23ce-6029-4ba9-9f9b-42913d7afc57 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:07.334236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:07.334895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:07.335203 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:07.335467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:07.390387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:07.399111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:07.462061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 809/1650] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:07.872135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:07.875758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:07.895187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdc0ab-2c2b-4585-8e36-22f10d8dd211 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:07.909620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:07.910624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:07.911073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:07.911572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] PATH_INFO: `/v3/projects/88ea7c290e7b4470b7b336ec9da930ca` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:07.913456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ea7c290e7b4470b7b336ec9da930ca)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:07.916387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:08.035111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ea7c290e7b4470b7b336ec9da930ca', 'request_id': 'req-494556ec-b086-443d-b0e8-2241067e8cb7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.051432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ea7c290e7b4470b7b336ec9da930ca', 'request_id': 'req-494556ec-b086-443d-b0e8-2241067e8cb7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.069256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ea7c290e7b4470b7b336ec9da930ca', 'request_id': 'req-494556ec-b086-443d-b0e8-2241067e8cb7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.105407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] The token cache is being invalidate because project 88ea7c290e7b4470b7b336ec9da930ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:08.106497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.109205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.115851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 842/1651] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:07 2022] DELETE /identity/v3/projects/88ea7c290e7b4470b7b336ec9da930ca => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:08.117013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:08.133042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89ca8-3ed1-4a4b-8c80-4ab44b4a014c service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:08.266420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:08.298920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:08.299619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:08.299898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:08.302819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] PATH_INFO: `/v3/users/5dc4e186c01142fba4fb6bd55d43e060` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:08.307584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc4e186c01142fba4fb6bd55d43e060)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:08.313802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:08.422313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc4e186c01142fba4fb6bd55d43e060'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.446869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc4e186c01142fba4fb6bd55d43e060'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.452954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc4e186c01142fba4fb6bd55d43e060'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.454789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc4e186c01142fba4fb6bd55d43e060'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.475024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bfb59359-b8d5-4c79-9be2-18cbc7794021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc4e186c01142fba4fb6bd55d43e060'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:08.517850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 810/1652] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:08 2022] DELETE /identity/v3/users/5dc4e186c01142fba4fb6bd55d43e060 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:29:09.116053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:09.118212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:09.141654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494556ec-b086-443d-b0e8-2241067e8cb7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:09.306488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:09.307251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:09.307484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:09.307762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] PATH_INFO: `/v3/projects/2c34199e1871487882981bb6a96a6e15` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:09.310040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c34199e1871487882981bb6a96a6e15)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:09.312894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:09.518410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:09.538174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c34199e1871487882981bb6a96a6e15', 'request_id': 'req-172bd010-5529-4477-93fb-51980fe0c19d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:09.575113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c34199e1871487882981bb6a96a6e15', 'request_id': 'req-172bd010-5529-4477-93fb-51980fe0c19d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:09.604288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c34199e1871487882981bb6a96a6e15', 'request_id': 'req-172bd010-5529-4477-93fb-51980fe0c19d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:09.656958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] The token cache is being invalidate because project 2c34199e1871487882981bb6a96a6e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:09.657829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:09.660965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-172bd010-5529-4477-93fb-51980fe0c19d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:09.680764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 843/1653] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:09 2022] DELETE /identity/v3/projects/2c34199e1871487882981bb6a96a6e15 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:09.682639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:09.683565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:09.684036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:09.744589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:09.745897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:10.011527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 811/1654] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:10.014117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.014637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.015122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.098959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:10.101085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:10.114993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 844/1655] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:10.120918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:10.161345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d76c4d-dfe4-4708-9464-89eb9da62d76 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:10.177302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:10.183124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.183861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.184558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.188277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:10.191686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:10.200233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b489-bb60-4792-b961-f3593d21bc1b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:10.262214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:10.267700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 812/1656] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:10.300221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff364ab8-de9e-487a-b15e-2f5b6d2bca43 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:10.317777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:10.318401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.321584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.322044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.322877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:10.326555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:10.347358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.348118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.348791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.397736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 845/1657] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:10 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:10.403719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:10.510493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] MFA Rules not processed for user `83593f8338c444b7b117eeaf3e4cdcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:10.513400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:10.548644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f679-c39b-48b6-9fc6-f497c02537e5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:10.555339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:10.556967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.557440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.557876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.559012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:10.561643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:10.597041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 846/1658] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:10 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:10.600315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:10.617463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:10.650480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a50cc-1a32-4fcb-9286-1b8ab05bce2e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:10.661471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:10.662126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.662571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.662849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] PATH_INFO: `/v3/projects/d9bf83e021ba49f5ac0b10b6d6157507` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.665460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:10.667438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9bf83e021ba49f5ac0b10b6d6157507)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:10.670245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:10.739292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 813/1659] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:10.742368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:10.781405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544463-80ef-4ca8-8215-74f153991673 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:10.801768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:10.803700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:10.805100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:10.805530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:10.807642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:10.810025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:10.842071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 814/1660] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:10.843389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:10.846858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:10.898112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9bf83e021ba49f5ac0b10b6d6157507', 'request_id': 'req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:10.941884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9bf83e021ba49f5ac0b10b6d6157507', 'request_id': 'req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:10.963365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442cec8-77e7-41e3-990a-250b2d9e6492 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:10.989758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9bf83e021ba49f5ac0b10b6d6157507', 'request_id': 'req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:11.028138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] The token cache is being invalidate because project d9bf83e021ba49f5ac0b10b6d6157507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:11.032190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:11.040222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:11.057732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 847/1661] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:10 2022] DELETE /identity/v3/projects/d9bf83e021ba49f5ac0b10b6d6157507 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:11.066900 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:11.071652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:11.075177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:11.077852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.078168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.078440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.088142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:11.146042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:11.275426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9beefa-9727-4bd0-99fd-b9f2bd6f8b81 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:11.281667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21637d3c-6827-47f1-9d46-3fd07795844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:11.282311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21637d3c-6827-47f1-9d46-3fd07795844d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.282584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21637d3c-6827-47f1-9d46-3fd07795844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.282858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21637d3c-6827-47f1-9d46-3fd07795844d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.283596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21637d3c-6827-47f1-9d46-3fd07795844d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:11.285659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21637d3c-6827-47f1-9d46-3fd07795844d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:11.300617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 848/1662] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:11.302076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813177e-0cee-497f-97d7-6b6a35e9e639 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.302558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813177e-0cee-497f-97d7-6b6a35e9e639 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.302720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813177e-0cee-497f-97d7-6b6a35e9e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.304584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:11.306653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:11.327936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 815/1663] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:11.328706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:11.339786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c813177e-0cee-497f-97d7-6b6a35e9e639 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:11.343619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c813177e-0cee-497f-97d7-6b6a35e9e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:11.357260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b31fbc-bd8e-4f56-86a1-a990ba540ab7 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:11.365744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:11.366852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.367267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.368257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] PATH_INFO: `/v3/projects/da49cbdc435c46748262ad9249b64261/users/4db84f162a224835bd15937257b44a92/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.375941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 849/1664] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:11.375941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.376438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.376438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.403407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:11.405116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:11.412191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:11.418989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 850/1665] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:11.419996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:11.424438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da49cbdc435c46748262ad9249b64261, user_id=4db84f162a224835bd15937257b44a92, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:11.429786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:11.432025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276c645-e72a-4b86-a029-f403694ac7af None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:11.440739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:11.443229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.443768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.444608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.446363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:11.447453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:11.454146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f1904-b31d-490a-9e5a-10d4b97b98d6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:11.477994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 816/1666] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:11 2022] PUT /identity/v3/projects/da49cbdc435c46748262ad9249b64261/users/4db84f162a224835bd15937257b44a92/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:11.492706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 851/1667] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:11.493387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:11.522446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.523257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.523749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.603858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] MFA Rules not processed for user `4db84f162a224835bd15937257b44a92`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:11.606577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:11.691897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e12042-05fc-4bd7-ac79-c4a20827f440 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:11.697192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:11.697988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:11.698326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:11.698706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:11.700682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:11.702994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:11.774636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 817/1668] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:11 2022] POST /identity/v3/projects => generated 363 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:29:11.779216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:11.921930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 852/1669] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:11.923818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:11.968591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5e86d-4b57-4509-bda8-8682cd65027f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.041597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00a1c9-ea00-4185-ace6-376ece820e77 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.063569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.065780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.067055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.067551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.074905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.080269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.165848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.168639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.169309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.172499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.172499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.172809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 818/1670] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:11 2022] POST /identity/v3/users => generated 282 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:12.174762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:12.177192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.216871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24925f9c-2c06-4ac3-b59e-b12f31373549 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.252875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.256668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.257774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.257774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.258800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.266856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.310986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 853/1671] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:11 2022] POST /identity/v3/projects => generated 373 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:12.317170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:12.340342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:12.345355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 819/1672] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:12.347352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:12.374394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1ab24-242f-42d1-9ad7-fb8c81debe49 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.596207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10409b12-279e-4e11-b838-6846279ec765 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.597625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:12.609973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.610662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.610936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.611205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.612761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.616325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.624844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.625656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.625939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.626204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.626867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.629991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.664374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 820/1673] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:12.677142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:12.690642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 854/1674] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:12 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:12.697072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:12.712732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68d487-715d-4054-960d-2a933fd57a13 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.725424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df07b51-5d9f-4db3-9146-dbb2e2a814b8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.729441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace92637-a4ad-45ea-aa58-87525287dc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.730559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace92637-a4ad-45ea-aa58-87525287dc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.731770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace92637-a4ad-45ea-aa58-87525287dc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.732884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace92637-a4ad-45ea-aa58-87525287dc8e admin admin] PATH_INFO: `/v3/projects/47f17b88cba14ba4bfe8982239bc9d8a/users/454d682545f54caa97ff5f0e435a0af0/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.740131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:12.740969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.742232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.742667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.744238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.752507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.783844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace92637-a4ad-45ea-aa58-87525287dc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f17b88cba14ba4bfe8982239bc9d8a, user_id=454d682545f54caa97ff5f0e435a0af0, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:12.784968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 855/1675] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:12.787806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace92637-a4ad-45ea-aa58-87525287dc8e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:12.790888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:12.819180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed42bee-ff5f-4b3d-90f0-332a590071f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:12.838181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 821/1676] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:12 2022] PUT /identity/v3/projects/47f17b88cba14ba4bfe8982239bc9d8a/users/454d682545f54caa97ff5f0e435a0af0/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:12.842472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:12.845972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:12.846722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:12.972354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] MFA Rules not processed for user `454d682545f54caa97ff5f0e435a0af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:12.974111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:13.098901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:13.102826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:13.103171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:13.103452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] PATH_INFO: `/v3/projects/e0fdcb9db5ad4a5ebfbfdbd5be0e7840/users/8bb4e0c3a5aa4a0ab995167fe082b5c4/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:13.149917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fdcb9db5ad4a5ebfbfdbd5be0e7840, user_id=8bb4e0c3a5aa4a0ab995167fe082b5c4, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:13.157233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:13.189462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 856/1677] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:12 2022] PUT /identity/v3/projects/e0fdcb9db5ad4a5ebfbfdbd5be0e7840/users/8bb4e0c3a5aa4a0ab995167fe082b5c4/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:13.191265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:13.211868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 822/1678] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:13.219424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:13.290782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5123ae-0f21-4289-8be5-5080d46d01b8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:13.292369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221ccd1-3bed-4bc6-a92d-73265170576e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:13.298855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:13.299802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:13.300088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:13.300359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:13.301059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:13.302863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:13.319893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 857/1679] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:13.325936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:13.335931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:13.340184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:13.354469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:13.355035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:13.366219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b60e4f-e291-404e-adc0-d8e08362f2a3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:13.375723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:13.376927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:13.388046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:13.393349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4618-911c-40f6-99ab-39bc499a73a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:13.398702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4618-911c-40f6-99ab-39bc499a73a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:13.399779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4618-911c-40f6-99ab-39bc499a73a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:13.400252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4618-911c-40f6-99ab-39bc499a73a1 admin admin] PATH_INFO: `/v3/projects/e0fdcb9db5ad4a5ebfbfdbd5be0e7840/users/8bb4e0c3a5aa4a0ab995167fe082b5c4/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:13.413186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a4618-911c-40f6-99ab-39bc499a73a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fdcb9db5ad4a5ebfbfdbd5be0e7840, user_id=8bb4e0c3a5aa4a0ab995167fe082b5c4, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:13.432644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a4618-911c-40f6-99ab-39bc499a73a1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:13.529794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 858/1680] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:13 2022] PUT /identity/v3/projects/e0fdcb9db5ad4a5ebfbfdbd5be0e7840/users/8bb4e0c3a5aa4a0ab995167fe082b5c4/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:13.535946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca09e-8099-4f0c-b07a-cb39c486b6d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:13.536429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca09e-8099-4f0c-b07a-cb39c486b6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:13.537239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca09e-8099-4f0c-b07a-cb39c486b6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:13.540911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 823/1681] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:13 2022] POST /identity/v3/projects => generated 365 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:13.547302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:13.739017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3cdca09e-8099-4f0c-b07a-cb39c486b6d4 None None] MFA Rules not processed for user `7c5ccd8c99934f60b4ca26c4990b7412`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:13.742035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3cdca09e-8099-4f0c-b07a-cb39c486b6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:13.951927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 859/1682] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:13.974387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:13.977183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:13.978450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:14.006070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c1710-ad34-442e-9a5c-150454cb46b4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:14.030014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5e0d0-9843-4095-97f1-0ae44a971425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:14.031005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5e0d0-9843-4095-97f1-0ae44a971425 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:14.031454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5e0d0-9843-4095-97f1-0ae44a971425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:14.031919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5e0d0-9843-4095-97f1-0ae44a971425 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:14.032961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5e0d0-9843-4095-97f1-0ae44a971425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:14.037733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5e0d0-9843-4095-97f1-0ae44a971425 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:14.109393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] MFA Rules not processed for user `8bb4e0c3a5aa4a0ab995167fe082b5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:14.114881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:14.160812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 824/1683] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:13 2022] POST /identity/v3/users => generated 289 bytes in 619 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:14.164973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:14.169309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:14.172030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:14.254149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] MFA Rules not processed for user `7c5ccd8c99934f60b4ca26c4990b7412`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:14.255084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:14.297435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 825/1684] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:14.300012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:14.346393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a55a-6ba6-4534-85de-4670e9962c12 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:14.351805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:14.352361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:14.352779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:14.352921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:14.353577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:14.355319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:14.384770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 826/1685] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:14.385724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:14.396107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:14.416384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:14.579905 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 860/1686] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 620 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:14.579905 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:14.602694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80dd2-f1d0-43e9-a457-372957bc1ea7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:14.622740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:14.623467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:14.623825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:14.624106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] PATH_INFO: `/v3/projects/49f23d9930de4488a2754245bfb0f439/users/45ba2be8f450494c9446ebbaf022b3e6/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:14.697141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f23d9930de4488a2754245bfb0f439, user_id=45ba2be8f450494c9446ebbaf022b3e6, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:14.710521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:14.763361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 861/1687] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:14 2022] PUT /identity/v3/projects/49f23d9930de4488a2754245bfb0f439/users/45ba2be8f450494c9446ebbaf022b3e6/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:14.764644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e30c5-91e8-4840-9b3a-dfbcea290b05 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:14.770004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:14.887234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:14.934483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19722010-3012-42ea-aaee-24847b9dd46c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:14.952237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:14.952978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:14.953427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:14.953547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:14.954216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:14.955874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:14.985406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 862/1688] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:14.987780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:14.993199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:15.040441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:15.041452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:15.041816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:15.042118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:15.287538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270c8c9-dbff-4d4c-8bd8-a8a8399bd5bb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:15.295642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:15.307960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:15.312617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b956d6-82b8-4ccc-91bc-df489ce8168f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:15.316302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b956d6-82b8-4ccc-91bc-df489ce8168f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:15.317442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b956d6-82b8-4ccc-91bc-df489ce8168f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:15.318466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b956d6-82b8-4ccc-91bc-df489ce8168f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:15.364701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 827/1689] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 979 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:15.366504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:15.449090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd0dda-d61c-4774-bed6-f47c648ca906 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:15.482012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:15.482012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:15.482012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:15.482012 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] PATH_INFO: `/v3/projects/49f23d9930de4488a2754245bfb0f439/users/45ba2be8f450494c9446ebbaf022b3e6/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:15.484299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f23d9930de4488a2754245bfb0f439, user_id=45ba2be8f450494c9446ebbaf022b3e6, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:15.494928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:15.556438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 828/1690] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:15 2022] PUT /identity/v3/projects/49f23d9930de4488a2754245bfb0f439/users/45ba2be8f450494c9446ebbaf022b3e6/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:15.558863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:15.567218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:15.646593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b956d6-82b8-4ccc-91bc-df489ce8168f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:15.649386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b956d6-82b8-4ccc-91bc-df489ce8168f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:15.691312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 863/1691] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:15.694578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa543b-750a-4670-ae34-e12c4431af32 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:15.695018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa543b-750a-4670-ae34-e12c4431af32 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:15.695483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa543b-750a-4670-ae34-e12c4431af32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:15.702670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:15.797837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-58fa543b-750a-4670-ae34-e12c4431af32 None None] MFA Rules not processed for user `45ba2be8f450494c9446ebbaf022b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:15.799702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-58fa543b-750a-4670-ae34-e12c4431af32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:15.832268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d0235-8b8f-4769-b766-9f535506ea71 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:15.854895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:15.862047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:15.862549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:15.865623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] PATH_INFO: `/v3/users/338cd3b7a7b24bbc83d448e508bfaef6` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:15.874833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338cd3b7a7b24bbc83d448e508bfaef6)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:15.880044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:15.960523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 864/1692] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:15.971263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e41f7-f271-4d02-89d8-ab1fccdc89d9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:15.975244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e41f7-f271-4d02-89d8-ab1fccdc89d9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:15.975582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e41f7-f271-4d02-89d8-ab1fccdc89d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:16.063415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:16.102497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338cd3b7a7b24bbc83d448e508bfaef6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:16.121094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338cd3b7a7b24bbc83d448e508bfaef6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:16.123518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338cd3b7a7b24bbc83d448e508bfaef6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:16.125627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a39e41f7-f271-4d02-89d8-ab1fccdc89d9 None None] MFA Rules not processed for user `8285583445bc4964a3b4b66b462d9212`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:16.127762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a39e41f7-f271-4d02-89d8-ab1fccdc89d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:16.129612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338cd3b7a7b24bbc83d448e508bfaef6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:16.164324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338cd3b7a7b24bbc83d448e508bfaef6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:16.231306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 829/1693] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:15 2022] DELETE /identity/v3/users/338cd3b7a7b24bbc83d448e508bfaef6 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:16.232660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:16.237680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:16.404715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1139d4b-6366-4599-b882-d5262dd44712 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:16.421299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:16.423793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:16.427712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:16.428622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:16.443298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:16.469570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 865/1694] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:16.474384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba0211-e950-4b5b-aa22-febe6ef5cd3e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:16.475139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba0211-e950-4b5b-aa22-febe6ef5cd3e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:16.475567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba0211-e950-4b5b-aa22-febe6ef5cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:16.541638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f7ba0211-e950-4b5b-aa22-febe6ef5cd3e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:16.546219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f7ba0211-e950-4b5b-aa22-febe6ef5cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:16.739032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 866/1695] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:16.740486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facbb79-8007-441e-ad92-a1400ff1a370 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:16.740778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facbb79-8007-441e-ad92-a1400ff1a370 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:16.741122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facbb79-8007-441e-ad92-a1400ff1a370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:16.785466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2facbb79-8007-441e-ad92-a1400ff1a370 None None] MFA Rules not processed for user `8285583445bc4964a3b4b66b462d9212`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:16.787011 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2facbb79-8007-441e-ad92-a1400ff1a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:16.842851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:16.865308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:16.872768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae56ec-aa20-44ec-aeac-4502c114f254 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:16.903798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 830/1696] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:16.906830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720880de-e154-401b-9d08-fa2d4c65a638 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:16.907566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720880de-e154-401b-9d08-fa2d4c65a638 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:16.908217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720880de-e154-401b-9d08-fa2d4c65a638 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:16.913571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 831/1697] 10.222.0.85 () {54 vars in 912 bytes} [Wed Nov 2 12:29:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:16.916128 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720880de-e154-401b-9d08-fa2d4c65a638 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:16.919694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-720880de-e154-401b-9d08-fa2d4c65a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:16.978730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 867/1698] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:16.986837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:16.987880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:16.988202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:17.156094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] MFA Rules not processed for user `e102dacae430417aa150ddd53cf34f31`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:17.160058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:17.220358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720880de-e154-401b-9d08-fa2d4c65a638 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:17.250982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:17.252167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:17.254950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:17.255498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:17.261989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:17.314455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:17.317974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:17.320185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 868/1699] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:17.321790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:17.324511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:17.393507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 832/1700] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:17.395043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:17.445097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2744ea-71e9-4a13-91e6-e350b0cb834e service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:17.476333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:17.477497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:17.477992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:17.478477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:17.485894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:17.575407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a38fb-4add-472f-8d90-363be81015d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:17.616752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:17.618163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:17.618614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:17.619063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:17.648036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:17.653919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:17.690075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 869/1701] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:17.692134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:17.739847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ad273-4153-40b9-986f-8e9425538f2f service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:17.741114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:17.743828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23b5c6-0b01-4900-b5a7-dc0e92117541 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:17.755425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:17.756687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:17.757215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:17.758176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:17.796287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 833/1702] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:17.798199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:17.801129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:17.840884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 870/1703] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:18.233527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:18.234035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:18.242027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:18.334177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] MFA Rules not processed for user `83593f8338c444b7b117eeaf3e4cdcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:18.335420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:18.617574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 834/1704] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:18.671333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:18.675123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:18.684949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:18.689298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:18.749909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736f48-91a9-45bc-a887-aa44c736094c service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:18.761135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:18.762709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:18.765184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:18.765477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] PATH_INFO: `/v3/projects/4052f9d3c4f340eba358014a59e54a11` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:18.768539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4052f9d3c4f340eba358014a59e54a11)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:18.772690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:18.795153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fd12b-068e-4e8d-9080-e15f3cc15ab1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:18.824944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:18.826075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:18.826550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:18.827028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] PATH_INFO: `/v3/users/499f80abd04b4580b0abbab52ce303e1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:18.842363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] RBAC: Authorizing `identity:delete_user(user_id=499f80abd04b4580b0abbab52ce303e1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:18.846804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:18.888845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4052f9d3c4f340eba358014a59e54a11', 'request_id': 'req-028feec0-3ff1-4350-8430-d520ae8f75c0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:18.899785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4052f9d3c4f340eba358014a59e54a11', 'request_id': 'req-028feec0-3ff1-4350-8430-d520ae8f75c0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:18.912939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4052f9d3c4f340eba358014a59e54a11', 'request_id': 'req-028feec0-3ff1-4350-8430-d520ae8f75c0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:18.932633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] The token cache is being invalidate because project 4052f9d3c4f340eba358014a59e54a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:18.934909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:18.937212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:18.941690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 871/1705] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:18 2022] DELETE /identity/v3/projects/4052f9d3c4f340eba358014a59e54a11 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:29:18.944238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:18.946529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.015979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:19.019799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:19.039607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499f80abd04b4580b0abbab52ce303e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.068256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028feec0-3ff1-4350-8430-d520ae8f75c0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:19.071959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499f80abd04b4580b0abbab52ce303e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.073207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499f80abd04b4580b0abbab52ce303e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.074955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:19.075464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499f80abd04b4580b0abbab52ce303e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.075998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.076407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.076868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] PATH_INFO: `/v3/users/092baf9aa02e48b08fdb2b2685ddaae2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.079286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=092baf9aa02e48b08fdb2b2685ddaae2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:19.082593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:19.092143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499f80abd04b4580b0abbab52ce303e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.132600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 835/1706] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:18 2022] DELETE /identity/v3/users/499f80abd04b4580b0abbab52ce303e1 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:19.139920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:19.144737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.179580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092baf9aa02e48b08fdb2b2685ddaae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.210530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092baf9aa02e48b08fdb2b2685ddaae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.215800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092baf9aa02e48b08fdb2b2685ddaae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.217713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092baf9aa02e48b08fdb2b2685ddaae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.246796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-361024c3-974e-4ace-9bdb-4bdd6925aa4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092baf9aa02e48b08fdb2b2685ddaae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.273287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 872/1707] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:18 2022] DELETE /identity/v3/users/092baf9aa02e48b08fdb2b2685ddaae2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:19.276379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fc557-79e0-4886-9d95-ad7206dc26b4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.276906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fc557-79e0-4886-9d95-ad7206dc26b4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.277816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fc557-79e0-4886-9d95-ad7206dc26b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.329620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-964fc557-79e0-4886-9d95-ad7206dc26b4 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:19.333508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-964fc557-79e0-4886-9d95-ad7206dc26b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.391459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b87bd7-2b95-4878-a9b0-8b99ccc9314d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:19.444824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 873/1708] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:19.446525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d36ba4-2511-4482-8962-0a70f57d94b7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.447158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d36ba4-2511-4482-8962-0a70f57d94b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.447499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d36ba4-2511-4482-8962-0a70f57d94b7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.454439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 874/1709] 10.222.0.85 () {54 vars in 912 bytes} [Wed Nov 2 12:29:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:19.457410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a150ce-5401-4542-8081-ec3e74dfcfbd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.457410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a150ce-5401-4542-8081-ec3e74dfcfbd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.457783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a150ce-5401-4542-8081-ec3e74dfcfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.501177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f3a150ce-5401-4542-8081-ec3e74dfcfbd None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:19.506077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3a150ce-5401-4542-8081-ec3e74dfcfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.543691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 875/1710] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:19.562816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.563359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.563854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.617970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:19.620475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.638033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 876/1711] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:19.639030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:19.640872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.651874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:19.653461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.653997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.654535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] PATH_INFO: `/v3/users/4db84f162a224835bd15937257b44a92` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.656921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db84f162a224835bd15937257b44a92)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:19.662653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:19.760796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:19.786859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db84f162a224835bd15937257b44a92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.802527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db84f162a224835bd15937257b44a92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.805689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db84f162a224835bd15937257b44a92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.807595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db84f162a224835bd15937257b44a92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.838882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a74fb568-e09b-42c0-a780-f74ff443dba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db84f162a224835bd15937257b44a92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:19.841632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:19.849653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3d748-108b-4a8c-83d5-9538d4087c3c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:19.870518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:19.872017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.872472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.873088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.884514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:19.905710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 836/1712] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:19 2022] DELETE /identity/v3/users/4db84f162a224835bd15937257b44a92 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:19.913817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:19.914318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:19.928463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:19.953640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:19.957673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1944953-fae7-43a0-ac2f-2d4267ee6f28 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:19.998553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 877/1713] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:20.000634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.001148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.001640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.003499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] MFA Rules not processed for user `e102dacae430417aa150ddd53cf34f31`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:20.006696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.028287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:20.030039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.211942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 878/1714] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:20.215079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.218998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.239416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 837/1715] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:20.245287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.251270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388504ed-d1bb-42a5-a9fe-b6bd1e82cc07 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.269643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.269643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.269643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.269643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.271149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.276066 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f42f3-cb5c-4cb4-bea3-950ed66cff89 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.291139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.292195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.292732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.293339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.297800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.300066 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.302759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f5e2-32a9-4125-a73b-8d70651f26ec service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.328091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.331074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.332599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 879/1716] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:20.334491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.337612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.337948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.351266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 838/1717] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:20.353180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.373244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f16021-ecda-43b1-baa2-5617cfd4f63e service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.385214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.386834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.387321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.387793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.388935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:20.389483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.397712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.403606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:20.404940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.425817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 880/1718] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:20.435231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.460579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 839/1719] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:20.464138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.472050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222f02d-1c19-4ca8-a261-99d6c29d39ef None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.478140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.478868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.479487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.479710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.500907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.502600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.507744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea01dbb-1077-4078-a26d-144460456432 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.524442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 881/1720] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:20.525578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.529797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.530796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.531266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.531419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:20.531744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.544789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.555404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.573260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ca17a-b2ab-4832-b51a-0a5cb2e64e42 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.584286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.585028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.585403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.585750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.632935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.641765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.655767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 840/1721] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:20 2022] POST /identity/v3/projects => generated 369 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:20.666069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.694573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 882/1722] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:20.839514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce8708-1e46-491c-908c-7c9801ba8d0e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.847682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.849395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.850096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.850850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.853221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.858196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:20.899129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 841/1723] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:20 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:20.903738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:20.916325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0476b5-c40f-4450-8887-aabb7eaae84e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:20.933106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:20.936959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:20.937150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:20.943082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:20.945614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:20.958829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:21.008808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 883/1724] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:21.014907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:21.071308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf6503-3d1c-497c-bc10-cb5febe93287 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:21.095231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2701c-8cad-4f81-9a00-cf2b14285c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:21.096382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2701c-8cad-4f81-9a00-cf2b14285c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:21.096877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2701c-8cad-4f81-9a00-cf2b14285c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:21.097439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2701c-8cad-4f81-9a00-cf2b14285c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:21.098302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2701c-8cad-4f81-9a00-cf2b14285c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:21.100732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2701c-8cad-4f81-9a00-cf2b14285c2d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:21.156959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 842/1725] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:21.163053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:21.178247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c8527-ab9d-471d-98cf-4773859ac5bf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:21.184355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d738ca-8a2b-4107-a0de-57dbf46f1a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:21.185083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d738ca-8a2b-4107-a0de-57dbf46f1a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:21.185366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d738ca-8a2b-4107-a0de-57dbf46f1a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:21.185642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d738ca-8a2b-4107-a0de-57dbf46f1a8f admin admin] PATH_INFO: `/v3/projects/0e38fb2a0d164791a7538cc9266f2cdc/users/488cbb96b0b2411e9261a9fd23d94266/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:21.237219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:21.237694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:21.237979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:21.251476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d738ca-8a2b-4107-a0de-57dbf46f1a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e38fb2a0d164791a7538cc9266f2cdc, user_id=488cbb96b0b2411e9261a9fd23d94266, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:21.260861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d738ca-8a2b-4107-a0de-57dbf46f1a8f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:21.327295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] MFA Rules not processed for user `7c5ccd8c99934f60b4ca26c4990b7412`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:21.330939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 884/1726] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:21 2022] PUT /identity/v3/projects/0e38fb2a0d164791a7538cc9266f2cdc/users/488cbb96b0b2411e9261a9fd23d94266/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:21.334802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:21.343368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:21.343657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:21.343979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:21.398481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:21.473595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] MFA Rules not processed for user `488cbb96b0b2411e9261a9fd23d94266`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:21.479231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:21.633084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 843/1727] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:21.697093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:21.803936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 885/1728] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:21.804452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:22.004807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce38b94-a4ef-4eb6-b99c-e534f800401a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:22.048349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:22.049445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:22.049939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:22.050425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:22.051633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:22.055420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:22.197887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 844/1729] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:21 2022] POST /identity/v3/projects => generated 367 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:22.203219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:22.224624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e4edf-b1f5-4628-ad4f-a51f29c33d8d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:22.532664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:22.535406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:22.536601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:22.537816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:22.541452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:22.565139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:22.589011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:22.598206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:22.676580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 886/1730] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:22 2022] POST /identity/v3/users => generated 290 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:22.685347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:22.689398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff399-401f-41e3-8262-be27e58602ad admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:22.693302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:22.730635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:22.732658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:22.739109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:22.739692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] PATH_INFO: `/v3/users/454d682545f54caa97ff5f0e435a0af0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:22.749350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=454d682545f54caa97ff5f0e435a0af0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:22.762345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:22.769135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f5b01-c6fc-4345-a087-78ef9a029612 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:22.795258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:22.799580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:22.805160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:22.806438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] PATH_INFO: `/v3/projects/36d6c3038bdf4479836310d68284de32` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:22.813732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d6c3038bdf4479836310d68284de32)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:22.825148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:22.980695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454d682545f54caa97ff5f0e435a0af0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.017909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454d682545f54caa97ff5f0e435a0af0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.019940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454d682545f54caa97ff5f0e435a0af0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.022709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454d682545f54caa97ff5f0e435a0af0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.097251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454d682545f54caa97ff5f0e435a0af0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.103176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d6c3038bdf4479836310d68284de32', 'request_id': 'req-ff278870-eb5f-4ae3-9d1c-f803f7232775'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.135378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d6c3038bdf4479836310d68284de32', 'request_id': 'req-ff278870-eb5f-4ae3-9d1c-f803f7232775'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.163016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d6c3038bdf4479836310d68284de32', 'request_id': 'req-ff278870-eb5f-4ae3-9d1c-f803f7232775'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.189204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 845/1731] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:22 2022] DELETE /identity/v3/users/454d682545f54caa97ff5f0e435a0af0 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:23.190993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:23.202440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:23.234080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] The token cache is being invalidate because project 36d6c3038bdf4479836310d68284de32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:23.234737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.238421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:23.243524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 887/1732] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:22 2022] DELETE /identity/v3/projects/36d6c3038bdf4479836310d68284de32 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:23.251596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:23.260456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:23.348206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:23.641303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d04d4-c4c5-4988-8829-d2da19930a7d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:23.672586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:23.675112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:23.676083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:23.676667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:23.677834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:23.685746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:23.688990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:23.733276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278870-eb5f-4ae3-9d1c-f803f7232775 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:23.779003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:23.781385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:23.784707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:23.785589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] PATH_INFO: `/v3/projects/5abb17741f2241d7b15d2c1b48b20f2b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:23.799309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5abb17741f2241d7b15d2c1b48b20f2b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:23.811354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:23.819277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 846/1733] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 625 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:23.819277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:23.841678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:23.924367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2001b3-b417-4854-b9cd-28d32e1baa10 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:23.966307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:23.968024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:23.968493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:23.968911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] PATH_INFO: `/v3/projects/da49cbdc435c46748262ad9249b64261` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:23.975330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=da49cbdc435c46748262ad9249b64261)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:23.977272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 12:29:23.977272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Nov 02 12:29:23.977272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: Nov 02 12:29:23.983231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:24.116934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5abb17741f2241d7b15d2c1b48b20f2b', 'request_id': 'req-b4b39bba-b56d-48f8-ae99-788b73a8a561'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.147295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5abb17741f2241d7b15d2c1b48b20f2b', 'request_id': 'req-b4b39bba-b56d-48f8-ae99-788b73a8a561'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.164828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5abb17741f2241d7b15d2c1b48b20f2b', 'request_id': 'req-b4b39bba-b56d-48f8-ae99-788b73a8a561'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.196360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] The token cache is being invalidate because project 5abb17741f2241d7b15d2c1b48b20f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:24.203946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.205545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4b39bba-b56d-48f8-ae99-788b73a8a561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.209920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 888/1734] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:23 2022] DELETE /identity/v3/projects/5abb17741f2241d7b15d2c1b48b20f2b => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:24.211653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:24.211981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:24.212759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:24.284926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da49cbdc435c46748262ad9249b64261', 'request_id': 'req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.285372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:24.289354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:24.349588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da49cbdc435c46748262ad9249b64261', 'request_id': 'req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.377373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da49cbdc435c46748262ad9249b64261', 'request_id': 'req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.453978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] The token cache is being invalidate because project da49cbdc435c46748262ad9249b64261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:24.455523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.460070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e300ba9c-aea0-43c3-9f88-0b27ee99fbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:24.483619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 847/1735] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:23 2022] DELETE /identity/v3/projects/da49cbdc435c46748262ad9249b64261 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:24.483789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 889/1736] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:24.483789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:24.485155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:24.500418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:24.500418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:24.500828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:24.510040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561dfa11-6277-4d8d-98a1-c1d0184c5807 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:24.533394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:24.534256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:24.534694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:24.535354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] PATH_INFO: `/v3/projects/5434e721e74f4b1b8ad8705c88b03189/users/0ee3afccc5664960b61bfd540cc0d173/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:24.561224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5434e721e74f4b1b8ad8705c88b03189, user_id=0ee3afccc5664960b61bfd540cc0d173, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:24.564710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:24.572460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:24.574822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:24.595537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 890/1737] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:24 2022] PUT /identity/v3/projects/5434e721e74f4b1b8ad8705c88b03189/users/0ee3afccc5664960b61bfd540cc0d173/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:24.598545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:24.602022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:24.954901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca29db-325e-49bb-9256-15a338cec0ec admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:24.961353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:24.962538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:24.963124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:24.963540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] PATH_INFO: `/v3/users/85f33f8322294446a683ef44edb8c631` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:24.966472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f33f8322294446a683ef44edb8c631)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:24.970665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:24.976517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:24.979361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 848/1738] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:24.985974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:24.992613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:25.089628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c183ca-413b-46bf-8ac5-ed539590a516 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:25.106495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:25.108031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:25.108744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:25.109560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] PATH_INFO: `/v3/users/8bb4e0c3a5aa4a0ab995167fe082b5c4` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:25.112494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb4e0c3a5aa4a0ab995167fe082b5c4)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:25.115736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:25.204757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f33f8322294446a683ef44edb8c631'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.232886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f33f8322294446a683ef44edb8c631'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.234398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f33f8322294446a683ef44edb8c631'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.237340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f33f8322294446a683ef44edb8c631'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.243881 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:25.276308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f33f8322294446a683ef44edb8c631'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.279560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb4e0c3a5aa4a0ab995167fe082b5c4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.291902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb4e0c3a5aa4a0ab995167fe082b5c4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.294126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb4e0c3a5aa4a0ab995167fe082b5c4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.295667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb4e0c3a5aa4a0ab995167fe082b5c4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.347686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 891/1739] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:24 2022] DELETE /identity/v3/users/85f33f8322294446a683ef44edb8c631 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:25.352949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:25.361511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:25.372179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb4e0c3a5aa4a0ab995167fe082b5c4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:25.447943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 849/1740] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:24 2022] DELETE /identity/v3/users/8bb4e0c3a5aa4a0ab995167fe082b5c4 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:25.455086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:25.460903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:25.646800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6362da1-0b38-4cf4-8cf5-152b34aab3a1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:25.691665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99e790-c494-4071-8f9e-6538ed5d3a99 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:25.698181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbf176-c5fa-46c8-824a-2d51d24306c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:25.699760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf176-c5fa-46c8-824a-2d51d24306c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:25.700248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf176-c5fa-46c8-824a-2d51d24306c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:25.700977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbf176-c5fa-46c8-824a-2d51d24306c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:25.701999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbf176-c5fa-46c8-824a-2d51d24306c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:25.704156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbf176-c5fa-46c8-824a-2d51d24306c9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:25.720606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 850/1741] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:25.722568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:25.723147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:25.723683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:25.763639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:25.765220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:25.787865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 851/1742] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:25.789047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:25.814414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65239cd2-1cf5-419e-b667-53b37dc96869 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:25.828305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:25.829667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:25.832234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:25.833237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] PATH_INFO: `/v3/projects/5434e721e74f4b1b8ad8705c88b03189/users/0ee3afccc5664960b61bfd540cc0d173/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:25.840334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5434e721e74f4b1b8ad8705c88b03189, user_id=0ee3afccc5664960b61bfd540cc0d173, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:25.849573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:25.859777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:25.865293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:25.867645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:25.872262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:25.883525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 852/1743] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:25 2022] PUT /identity/v3/projects/5434e721e74f4b1b8ad8705c88b03189/users/0ee3afccc5664960b61bfd540cc0d173/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:25.885139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:25.885566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:25.990720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:25.998221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382aa05-7864-4ac0-b256-4d5d5739b845 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:26.037547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 892/1744] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:26.039400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:26.040726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:26.041373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:26.075539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d313b-b75c-48ff-a66c-feb247cb4972 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:26.086802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:26.087835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:26.089208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:26.089627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:26.092549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:26.096117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:26.098562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:26.130685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 853/1745] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:26.133332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:26.156984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ef600-30ff-4215-8cfc-3c478621cfd2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:26.174386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976dc1e-511f-4e77-8b5a-9f3ac98cb038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:26.179502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976dc1e-511f-4e77-8b5a-9f3ac98cb038 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:26.179502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976dc1e-511f-4e77-8b5a-9f3ac98cb038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:26.182678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976dc1e-511f-4e77-8b5a-9f3ac98cb038 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:26.183626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976dc1e-511f-4e77-8b5a-9f3ac98cb038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:26.188578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] MFA Rules not processed for user `0ee3afccc5664960b61bfd540cc0d173`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:26.192508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976dc1e-511f-4e77-8b5a-9f3ac98cb038 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:26.193384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:26.301726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:26.315902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 854/1746] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:26 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:29:26.315902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085103e-7f18-414b-9639-acea9796233c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:26.315902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085103e-7f18-414b-9639-acea9796233c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:26.315902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085103e-7f18-414b-9639-acea9796233c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:26.353059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:26.423623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 893/1747] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:26.424621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:26.441437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340041d3-2f18-4a5f-9b7a-38a2d58ba24f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:26.532944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1085103e-7f18-414b-9639-acea9796233c None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:26.534470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1085103e-7f18-414b-9639-acea9796233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:26.681535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:26.685689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:26.686067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:26.686363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:26.691587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:26.705274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:26.809875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 855/1748] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:26.811569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085103e-7f18-414b-9639-acea9796233c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:26.817868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 894/1749] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:26 2022] POST /identity/v3/users => generated 287 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:26.818367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1085103e-7f18-414b-9639-acea9796233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:26.819350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:26.871856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ed3dc-2f7d-435a-a157-2e135e78461e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:26.886442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:26.887146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:26.887343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:26.887619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:26.891940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:26.895143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:26.949948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 895/1750] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:26.960408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:26.986240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085103e-7f18-414b-9639-acea9796233c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:27.001870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:27.003091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:27.003513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:27.003959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:27.004292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bd9d6-1f69-48af-a75f-82d04492609e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:27.022680 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:27.028482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:27.030663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:27.031212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:27.033731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:27.037036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:27.076166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 896/1751] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:27.079849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:27.121104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adf256-1cb3-456a-bb9d-96d7c3665c63 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:27.137135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:27.139418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:27.142024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:27.143195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:27.143696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:27.144174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] PATH_INFO: `/v3/projects/bc387749af3b48729d467782a55262d3/users/557e0ee320e84adeb03ccdd4f8559577/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:27.157068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 856/1752] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:27.158237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:27.159977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:27.234140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:27.254888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc387749af3b48729d467782a55262d3, user_id=557e0ee320e84adeb03ccdd4f8559577, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:27.268107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:27.314200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 897/1753] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:27 2022] PUT /identity/v3/projects/bc387749af3b48729d467782a55262d3/users/557e0ee320e84adeb03ccdd4f8559577/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:27.315214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:27.328435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:27.543485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0395c-f627-49f0-8199-a8bda74ce207 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:27.737448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70937781-ae41-4ae8-ab33-ccdbc87c9de9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:27.780517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:27.787704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:27.789435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:27.791915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] PATH_INFO: `/v3/projects/cca31d41c4894131b2cdd269c6dc26c8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:27.798663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca31d41c4894131b2cdd269c6dc26c8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:27.812779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:27.815444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977e133-82ec-4cea-9f42-143d88d39ed0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:27.826610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977e133-82ec-4cea-9f42-143d88d39ed0 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:27.826875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977e133-82ec-4cea-9f42-143d88d39ed0 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:27.827286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977e133-82ec-4cea-9f42-143d88d39ed0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:27.891198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:28.033474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca31d41c4894131b2cdd269c6dc26c8', 'request_id': 'req-9d3aa5a4-4147-400e-a3d6-600ef776d024'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:28.069593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca31d41c4894131b2cdd269c6dc26c8', 'request_id': 'req-9d3aa5a4-4147-400e-a3d6-600ef776d024'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:28.090091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca31d41c4894131b2cdd269c6dc26c8', 'request_id': 'req-9d3aa5a4-4147-400e-a3d6-600ef776d024'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:28.178510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] The token cache is being invalidate because project cca31d41c4894131b2cdd269c6dc26c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:28.178993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:28.186707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d3aa5a4-4147-400e-a3d6-600ef776d024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:28.192950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 898/1754] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:27 2022] DELETE /identity/v3/projects/cca31d41c4894131b2cdd269c6dc26c8 => generated 0 bytes in 879 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:28.201434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977e133-82ec-4cea-9f42-143d88d39ed0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:28.203567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977e133-82ec-4cea-9f42-143d88d39ed0 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:28.210825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:28.211282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:28.211742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:28.228422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 857/1755] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:29:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1070 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:29:28.234017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379bf6-abbb-4ac5-b515-b9f3fe504669 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:28.234918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379bf6-abbb-4ac5-b515-b9f3fe504669 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:28.235837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be379bf6-abbb-4ac5-b515-b9f3fe504669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:28.333710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-be379bf6-abbb-4ac5-b515-b9f3fe504669 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:28.341139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-be379bf6-abbb-4ac5-b515-b9f3fe504669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:28.346616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] MFA Rules not processed for user `557e0ee320e84adeb03ccdd4f8559577`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:28.348786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:28.619033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:28.717191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 858/1756] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:28.731573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:28.732155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:28.732599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:28.838059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:28.844645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:28.855747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 899/1757] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 662 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:28.906519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 859/1758] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:29:28.916610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:28.975689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41f95a-1b3b-470b-8e79-768914b7ecd4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:29.057031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:29.057031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:29.057031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:29.057031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:29.057967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:29.057967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:29.063514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:29.123245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:29.125654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:29.187485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 900/1759] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:29.195049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:29.233987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27eb272-8880-4246-995b-e8785c5867fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:29.250424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:29.252451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:29.253371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:29.254211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:29.255801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:29.265526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:29.373933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 901/1760] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:29 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:29:29.380107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:29.792946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18567905-7243-45b2-bad1-fc8c71669867 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:29.806226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:29.936519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0b7be-7af1-4e55-add5-7ad1c87e69fe admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:29.954662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:29.956061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:29.956491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:29.956906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:29.957785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:29.962901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:30.091961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 902/1761] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:29 2022] POST /identity/v3/users => generated 294 bytes in 695 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:30.103991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:30.114814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:30.141433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:30.141433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:30.142824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:30.147194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:30.200522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dcc75-75ce-4d9e-bbfb-6722998f55c6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:30.223673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:30.225180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:30.227232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:30.227942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] PATH_INFO: `/v3/projects/47f17b88cba14ba4bfe8982239bc9d8a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:30.237350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f17b88cba14ba4bfe8982239bc9d8a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:30.249634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:30.439773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:30.442905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:30.444613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f17b88cba14ba4bfe8982239bc9d8a', 'request_id': 'req-5bf96480-27d2-400e-9602-bfb56c3a63ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:30.481747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f17b88cba14ba4bfe8982239bc9d8a', 'request_id': 'req-5bf96480-27d2-400e-9602-bfb56c3a63ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:30.482596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 860/1762] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 1367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:30.484328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:30.498548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:30.515284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f17b88cba14ba4bfe8982239bc9d8a', 'request_id': 'req-5bf96480-27d2-400e-9602-bfb56c3a63ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:30.546872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] The token cache is being invalidate because project 47f17b88cba14ba4bfe8982239bc9d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:30.547285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:30.547951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:30.553545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 903/1763] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:30 2022] DELETE /identity/v3/projects/47f17b88cba14ba4bfe8982239bc9d8a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:30.553545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:30.558142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:30.594828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7532af-23ee-47b1-955a-c17c39345df6 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:30.668619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96480-27d2-400e-9602-bfb56c3a63ee admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:30.693377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:30.705471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:30.706117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:30.706392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:30.711591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:30.712495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:30.718153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:30.751949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 904/1764] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:30.752898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:30.759038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:30.824695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83579fb4-3d4e-49c9-9f0d-81d2d0cc4927 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:30.861033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:30.861747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:30.862045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:30.862314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] PATH_INFO: `/v3/users/45ba2be8f450494c9446ebbaf022b3e6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:30.863607 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ba2be8f450494c9446ebbaf022b3e6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:30.866076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:30.971751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:30.972863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:30.973691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:30.974595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] PATH_INFO: `/v3/projects/e0fdcb9db5ad4a5ebfbfdbd5be0e7840` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:30.983092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fdcb9db5ad4a5ebfbfdbd5be0e7840)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:30.988757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:31.044971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ba2be8f450494c9446ebbaf022b3e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.071931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ba2be8f450494c9446ebbaf022b3e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.072636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ba2be8f450494c9446ebbaf022b3e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.085753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ba2be8f450494c9446ebbaf022b3e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.093851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:31.109458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ba2be8f450494c9446ebbaf022b3e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.190621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 905/1765] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:30 2022] DELETE /identity/v3/users/45ba2be8f450494c9446ebbaf022b3e6 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:31.192391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:31.205388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:31.351586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fdcb9db5ad4a5ebfbfdbd5be0e7840', 'request_id': 'req-aae0e712-1b54-4a08-b717-372e00da3526'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.406877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fdcb9db5ad4a5ebfbfdbd5be0e7840', 'request_id': 'req-aae0e712-1b54-4a08-b717-372e00da3526'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.433810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fdcb9db5ad4a5ebfbfdbd5be0e7840', 'request_id': 'req-aae0e712-1b54-4a08-b717-372e00da3526'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.493624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] The token cache is being invalidate because project e0fdcb9db5ad4a5ebfbfdbd5be0e7840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:31.494740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.497037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aae0e712-1b54-4a08-b717-372e00da3526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:31.507081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 861/1766] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:30 2022] DELETE /identity/v3/projects/e0fdcb9db5ad4a5ebfbfdbd5be0e7840 => generated 0 bytes in 1017 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:31.511201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:31.512411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:31.513154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:31.556296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:31.559925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:31.569614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa283eb0-1f08-4aff-a472-731d4e955323 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:31.855789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:31.856884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:31.857368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:31.860397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:31.862184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:31.866394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:31.897253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 906/1767] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 708 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:31.904553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:31.908193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:31.935951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bf3dc-12dc-453a-8894-34e2a04b02fc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:31.946157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:31.954311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:31.954311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:31.954311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] PATH_INFO: `/v3/users/557e0ee320e84adeb03ccdd4f8559577` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:31.954311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=557e0ee320e84adeb03ccdd4f8559577)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:31.955342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:31.983810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 862/1768] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:31.986733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:31.990504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:32.046975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258bb71-3a2d-435d-955a-f2c6d5256fa8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:32.061049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4753b5e-72f6-4d1a-9e14-ccbb5b5f5e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:32.063312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4753b5e-72f6-4d1a-9e14-ccbb5b5f5e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:32.064230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4753b5e-72f6-4d1a-9e14-ccbb5b5f5e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:32.065127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557e0ee320e84adeb03ccdd4f8559577'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:32.067785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4753b5e-72f6-4d1a-9e14-ccbb5b5f5e35 admin admin] PATH_INFO: `/v3/projects/f6078a77309a4c30a68ec13a058d6948/users/1be795c872844dda9ce8c39b65840685/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:32.087088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557e0ee320e84adeb03ccdd4f8559577'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:32.089380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557e0ee320e84adeb03ccdd4f8559577'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:32.092734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557e0ee320e84adeb03ccdd4f8559577'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:32.111319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a2f4d52-307d-4f54-a527-b53446f4a2a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557e0ee320e84adeb03ccdd4f8559577'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:32.132692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4753b5e-72f6-4d1a-9e14-ccbb5b5f5e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6078a77309a4c30a68ec13a058d6948, user_id=1be795c872844dda9ce8c39b65840685, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:32.148351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:32.156743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4753b5e-72f6-4d1a-9e14-ccbb5b5f5e35 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:32.191856 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 907/1769] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:31 2022] DELETE /identity/v3/users/557e0ee320e84adeb03ccdd4f8559577 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:32.195245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8c061-547d-46d3-9213-34300835c655 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:32.195859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8c061-547d-46d3-9213-34300835c655 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:32.196169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8c061-547d-46d3-9213-34300835c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:32.226776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 863/1770] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:31 2022] PUT /identity/v3/projects/f6078a77309a4c30a68ec13a058d6948/users/1be795c872844dda9ce8c39b65840685/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:32.229978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6168b-93c5-49f2-b4f1-495f7e0099ae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:32.230481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6168b-93c5-49f2-b4f1-495f7e0099ae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:32.230973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6168b-93c5-49f2-b4f1-495f7e0099ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:32.298333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-70e8c061-547d-46d3-9213-34300835c655 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:32.301498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-70e8c061-547d-46d3-9213-34300835c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:32.321851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-92c6168b-93c5-49f2-b4f1-495f7e0099ae None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:32.322938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-92c6168b-93c5-49f2-b4f1-495f7e0099ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:32.367565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:32.491160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 864/1771] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:32.494157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c5f9-4d4e-424e-af5f-1d4881a181cb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:32.494588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c5f9-4d4e-424e-af5f-1d4881a181cb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:32.495979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c5f9-4d4e-424e-af5f-1d4881a181cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:32.622232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3c77c5f9-4d4e-424e-af5f-1d4881a181cb None None] MFA Rules not processed for user `1be795c872844dda9ce8c39b65840685`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:32.626935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c77c5f9-4d4e-424e-af5f-1d4881a181cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:32.740802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 908/1772] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:32.750085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8c061-547d-46d3-9213-34300835c655 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:32.752426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-70e8c061-547d-46d3-9213-34300835c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:32.909349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 865/1773] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:32.920701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:32.921229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:32.921752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.022787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] MFA Rules not processed for user `e102dacae430417aa150ddd53cf34f31`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:33.025206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:33.049922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 866/1774] 10.222.0.87 () {58 vars in 1015 bytes} [Wed Nov 2 12:29:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:33.051564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:33.053771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:33.061507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8c061-547d-46d3-9213-34300835c655 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:33.082271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:33.085160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.085748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.087644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.121753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a1658-9272-4205-9ecb-de2f9ddecb0a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:33.135232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:33.137932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.138985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.139622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] PATH_INFO: `/v3/users/488cbb96b0b2411e9261a9fd23d94266` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.141444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] RBAC: Authorizing `identity:delete_user(user_id=488cbb96b0b2411e9261a9fd23d94266)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:33.142577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:33.144667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:33.152755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:33.213912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 909/1775] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:33.219261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:33.228595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '488cbb96b0b2411e9261a9fd23d94266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:33.241068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '488cbb96b0b2411e9261a9fd23d94266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:33.242329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '488cbb96b0b2411e9261a9fd23d94266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:33.243574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '488cbb96b0b2411e9261a9fd23d94266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:33.256186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '488cbb96b0b2411e9261a9fd23d94266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:33.274899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a389e-2db1-4ea3-8b35-7396005d0927 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:33.283935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 867/1776] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:33 2022] DELETE /identity/v3/users/488cbb96b0b2411e9261a9fd23d94266 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:33.288923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:33.290740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:33.325068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:33.429796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e56b-8135-4cb3-9ae8-d3a4c6c2044d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:33.438830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:33.439876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.440270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.441512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.444608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:33.447125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:33.493087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 868/1777] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:33 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:33.493087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:33.500539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:33.583309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:33.584377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.584858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.585305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.586621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:33.587464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:33.601419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb75a7e-4c46-4973-add2-397032d54041 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:33.605821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1636a-294f-4c2a-b2c6-bc8b708c862a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:33.619872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:33.621579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.622540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.622994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.626204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:33.646797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:33.647056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 910/1778] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:33.650393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:33.654944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f0423-a306-4933-843f-39c2575660ae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.655419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f0423-a306-4933-843f-39c2575660ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.655860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f0423-a306-4933-843f-39c2575660ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.682572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 869/1779] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:33.684922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:33.757917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-af5f0423-a306-4933-843f-39c2575660ae None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:33.762883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af5f0423-a306-4933-843f-39c2575660ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:33.841481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:33.944003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32d037-271f-4c44-9baf-db446b64066b service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:33.970194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:33.974528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:33.978292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:33.978768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:33.981993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:33.985901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.027081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 911/1780] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:34.029671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f0423-a306-4933-843f-39c2575660ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.033783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af5f0423-a306-4933-843f-39c2575660ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:34.057823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 870/1781] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:33 2022] POST /identity/v3/users => generated 299 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:34.066147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.068042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f0423-a306-4933-843f-39c2575660ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.089603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.093284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba5a0-d540-414e-94fb-6664b99599a0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.093519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.094774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.095187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.095998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.098412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.107867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.108716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.109359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.109756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.110664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.113750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.136478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 871/1782] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:34.138954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.191588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74727e6-aea0-4df9-b56f-196e02971ff4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.199636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.200467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.201151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.201558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] PATH_INFO: `/v3/projects/1c2a56b4d5d7410e8b512d3622b61a61/users/dafea32b02024e99a95dc7eb1b47457e/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.211135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 912/1783] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:34 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:34.218429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.247607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2a56b4d5d7410e8b512d3622b61a61, user_id=dafea32b02024e99a95dc7eb1b47457e, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.252235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.284774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 872/1784] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:34 2022] PUT /identity/v3/projects/1c2a56b4d5d7410e8b512d3622b61a61/users/dafea32b02024e99a95dc7eb1b47457e/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:34.286119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.288886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:34.296889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:34.377867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86caa0d-86fb-4506-8c3b-3dc36602f6d4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.384359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.385236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.386375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.386834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.508429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.510334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.527140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 873/1785] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:34.528351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.551626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae2bbb-cbae-485c-9f93-2ec2bb99b470 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.561744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.562880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.563286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.563665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.564524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.566568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.593586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7ccd7-44a5-42c8-bcbe-1891a5f0b810 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.597363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 874/1786] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:34.627511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.629343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.630422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.630725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.635286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.636401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.644254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.703823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a201d3-1425-4148-8313-d8770ea26e69 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.734396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeac460-3a57-4d45-84eb-9436d2197942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.736355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeac460-3a57-4d45-84eb-9436d2197942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.736868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeac460-3a57-4d45-84eb-9436d2197942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.737433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeac460-3a57-4d45-84eb-9436d2197942 admin admin] PATH_INFO: `/v3/projects/1c2a56b4d5d7410e8b512d3622b61a61/users/dafea32b02024e99a95dc7eb1b47457e/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.742647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 913/1787] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:34 2022] POST /identity/v3/users => generated 283 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:34.749466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.776656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeac460-3a57-4d45-84eb-9436d2197942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2a56b4d5d7410e8b512d3622b61a61, user_id=dafea32b02024e99a95dc7eb1b47457e, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.781421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced5e3-d939-4428-9258-edc25744d388 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.786552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeac460-3a57-4d45-84eb-9436d2197942 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.787548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:34.788175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.788854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.789305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.790110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:34.792617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:34.815319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 914/1788] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:34.825190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:34.849765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6229134-bc8d-47cd-8f4e-29c81c4d54d0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:34.856230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 875/1789] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:34 2022] PUT /identity/v3/projects/1c2a56b4d5d7410e8b512d3622b61a61/users/dafea32b02024e99a95dc7eb1b47457e/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:34.881752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:34.882475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:34.883414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:34.976355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] MFA Rules not processed for user `dafea32b02024e99a95dc7eb1b47457e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:34.978384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:35.085927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:35.087965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:35.090009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:35.090594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:35.094205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:35.101731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:35.141986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 915/1790] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:35.141986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:35.152445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:35.181296 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbbe78-237a-46f2-bbf0-722cb15b7c89 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:35.194216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:35.195956 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:35.196316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:35.196683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] PATH_INFO: `/v3/projects/49f23d9930de4488a2754245bfb0f439` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:35.200402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 876/1791] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:35.201465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:35.206949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f23d9930de4488a2754245bfb0f439)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:35.209054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:35.235899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a0fc-7742-4e05-b929-001e2853b3ce None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:35.258051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:35.265926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:35.267602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:35.269646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] PATH_INFO: `/v3/projects/f4cd775156864b8689cd8b4fe864e2be/users/3e0c3322cc6e494db65980cf309f3e0a/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:35.295767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:35.332591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:35.345661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4cd775156864b8689cd8b4fe864e2be, user_id=3e0c3322cc6e494db65980cf309f3e0a, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:35.367981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:35.403510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 877/1792] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:35 2022] PUT /identity/v3/projects/f4cd775156864b8689cd8b4fe864e2be/users/3e0c3322cc6e494db65980cf309f3e0a/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:35.404468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:35.406429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:35.434194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f23d9930de4488a2754245bfb0f439', 'request_id': 'req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:35.450286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f23d9930de4488a2754245bfb0f439', 'request_id': 'req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:35.467179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f23d9930de4488a2754245bfb0f439', 'request_id': 'req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:35.500764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] The token cache is being invalidate because project 49f23d9930de4488a2754245bfb0f439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:35.501599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:35.503262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bfb9eeb-1f53-4498-af9e-04f472ff84b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:35.509046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 916/1793] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:35 2022] DELETE /identity/v3/projects/49f23d9930de4488a2754245bfb0f439 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:35.518138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:35.519576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:35.520181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:35.577858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b701-d414-484a-98d0-6ec690630ce0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:35.606392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:35.607931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:35.608342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:35.608945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:35.611980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:35.641760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] MFA Rules not processed for user `3e0c3322cc6e494db65980cf309f3e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:35.644365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:35.842598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:35.846403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fb65d-d246-4cd9-9557-0f736acd0342 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:35.879001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 878/1794] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:35.884149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:35.884830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:35.885766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:35.948330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:35.951147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.022457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 917/1795] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:36.033725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:36.037274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.129331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 879/1796] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:36.129331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:36.137763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.192306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9319c-67f6-4df8-9b3b-12583247195b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:36.208768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:36.209582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:36.209862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:36.210684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:36.211902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:36.214542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:36.287801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 880/1797] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:36 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:36.288800 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:36.292819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.349949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:36.354703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324c6d6-6d7c-4e61-a884-d02c9ecd8e5b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:36.439173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06a82-d140-4483-9a18-e1946334e6e0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:36.446294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:36.446959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:36.447452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:36.448007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] PATH_INFO: `/v3/projects/bc387749af3b48729d467782a55262d3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:36.449896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc387749af3b48729d467782a55262d3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:36.454373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:36.487359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:36.539345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc387749af3b48729d467782a55262d3', 'request_id': 'req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.547144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc387749af3b48729d467782a55262d3', 'request_id': 'req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.558961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc387749af3b48729d467782a55262d3', 'request_id': 'req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.562659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:36.563787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:36.564282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:36.564763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] PATH_INFO: `/v3/users/1be795c872844dda9ce8c39b65840685` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:36.570510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be795c872844dda9ce8c39b65840685)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:36.574569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:36.587298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] The token cache is being invalidate because project bc387749af3b48729d467782a55262d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:36.587659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.588579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f08eac0a-5765-4a4d-8df4-971e1fe5e69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.594378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 881/1798] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:36 2022] DELETE /identity/v3/projects/bc387749af3b48729d467782a55262d3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:36.596476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:36.596749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:36.597024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:36.623807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:36.626385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.787062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be795c872844dda9ce8c39b65840685'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.796142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 882/1799] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:36.796142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:36.801607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.813110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be795c872844dda9ce8c39b65840685'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.814146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be795c872844dda9ce8c39b65840685'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.815256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be795c872844dda9ce8c39b65840685'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.836522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96893da-54fd-4308-86d3-56ac5ffa794d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be795c872844dda9ce8c39b65840685'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:36.859687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183db5b-b4d7-441b-8759-aaa9f713ba06 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:36.882700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:36.885161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:36.886566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:36.888566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:36.894220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:36.904316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 918/1800] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:36 2022] DELETE /identity/v3/users/1be795c872844dda9ce8c39b65840685 => generated 0 bytes in 879 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:36.908175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:36.908599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:36.909222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:36.909711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:36.973461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 883/1801] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:36 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:36.975103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:36.977690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:36.998736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf76cad-8f88-4c45-9ba7-b83a76d2d65f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:37.002357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:37.010431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:37.054016 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:37.124518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:37.125454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.128174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.143322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.144765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:37.147123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:37.152626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462a3f2-60f8-4472-972a-87f723ae75c1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:37.202496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 884/1802] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:37.207394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.208041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.208458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.250666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 919/1803] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:37.251762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.258798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:37.263927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:37.268733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:37.290130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 885/1804] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:37.291155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840237d-1f2a-4254-93f0-7c477956ada4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:37.294613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.313073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aace960-28b4-4746-a1bd-c35a7127e0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:37.313719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aace960-28b4-4746-a1bd-c35a7127e0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.314050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aace960-28b4-4746-a1bd-c35a7127e0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.314330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aace960-28b4-4746-a1bd-c35a7127e0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.314969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aace960-28b4-4746-a1bd-c35a7127e0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:37.316679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aace960-28b4-4746-a1bd-c35a7127e0d4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:37.324098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ede4cd-f12e-4e0b-b0f8-71d313cc1355 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:37.342802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:37.343437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.343714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.343991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.344757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:37.347973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 920/1805] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:37.349491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.349758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.350038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.350468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:37.397454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:37.399664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:37.425519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 921/1806] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:37.426675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.438721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 886/1807] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:37 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:37.440978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.443996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:37.455796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e24e8-5af5-4266-864a-a9d18d35fbb8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:37.566568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:37.627646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:37.628382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.628660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.628936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] PATH_INFO: `/v3/projects/66677dc61a194ea68885ddac59bb2579/users/1bdd227aff83453bac3abf9dfb291b30/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.630589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818c46-ddc9-40c9-a0d0-d8bc2d3a4107 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:37.638616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:37.639630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.640153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.640829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.656237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:37.658426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:37.674293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66677dc61a194ea68885ddac59bb2579, user_id=1bdd227aff83453bac3abf9dfb291b30, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:37.677109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:37.679427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 887/1808] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:37.680598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.697530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc3d26-4a5c-4793-a127-b5733b2f0cd6 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:37.706726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:37.707452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.707920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.708250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.709869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:37.710332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:37.715954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89d950-6713-4742-bb93-5e43ef9ff1ed admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:37.720055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 922/1809] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:37 2022] PUT /identity/v3/projects/66677dc61a194ea68885ddac59bb2579/users/1bdd227aff83453bac3abf9dfb291b30/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:37.721113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.747601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 888/1810] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:37.749621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f044307-ab00-4de0-bff1-cd0737e4b61d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:37.750501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f044307-ab00-4de0-bff1-cd0737e4b61d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:37.751392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f044307-ab00-4de0-bff1-cd0737e4b61d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:37.754352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 889/1811] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:29:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:37.755562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f044307-ab00-4de0-bff1-cd0737e4b61d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:37.961648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba287204-af77-470d-af86-911541aef5f3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.001025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e0a51-e5d6-475d-9d04-c46b63ed3081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.001687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e0a51-e5d6-475d-9d04-c46b63ed3081 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.002213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e0a51-e5d6-475d-9d04-c46b63ed3081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.002699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e0a51-e5d6-475d-9d04-c46b63ed3081 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.003907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e0a51-e5d6-475d-9d04-c46b63ed3081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.005183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f044307-ab00-4de0-bff1-cd0737e4b61d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.008219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e0a51-e5d6-475d-9d04-c46b63ed3081 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.012621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.014038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.014374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.014602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.015345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.026753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.060357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 890/1812] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:38.062717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.065470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 923/1813] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:37 2022] POST /identity/v3/users => generated 285 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:38.070100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.081790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.082283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.089870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5927-525f-4621-957e-160debb01dd3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.102912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.104984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.105320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.110775 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.113120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.116140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.129783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:38.131984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:38.182322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:38.206670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 891/1814] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:38 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:38.207969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.397798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 924/1815] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:38.399144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.415590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4523-4a00-4a49-b925-fda266e416c8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.426555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.429904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.430476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.430763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] PATH_INFO: `/v3/projects/66677dc61a194ea68885ddac59bb2579/users/1bdd227aff83453bac3abf9dfb291b30/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.433613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363ee37-346a-48f7-a032-d30f06e42404 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.433823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66677dc61a194ea68885ddac59bb2579, user_id=1bdd227aff83453bac3abf9dfb291b30, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.438292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.452400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbe658-61fe-4afb-9933-5ee387fcf7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.455345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe658-61fe-4afb-9933-5ee387fcf7df admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.456624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe658-61fe-4afb-9933-5ee387fcf7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.457491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 892/1816] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:38 2022] PUT /identity/v3/projects/66677dc61a194ea68885ddac59bb2579/users/1bdd227aff83453bac3abf9dfb291b30/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:38.458233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe658-61fe-4afb-9933-5ee387fcf7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.458662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.459542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbe658-61fe-4afb-9933-5ee387fcf7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.475300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbe658-61fe-4afb-9933-5ee387fcf7df admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.502737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 925/1817] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:38.506800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.506800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.506800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.551644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] MFA Rules not processed for user `1bdd227aff83453bac3abf9dfb291b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:38.553303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:38.593977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:38.619364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf4aec-31e3-4a79-8eaf-b3f86c66cc8e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.634444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.635098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.635531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.637927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.642693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.645253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.704328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 893/1818] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:38 2022] POST /identity/v3/users => generated 282 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:38.705653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.713484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:38.732906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 926/1819] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:38.736644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.754278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668f3db-c808-47b3-bc8c-351f17806565 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.761501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.762573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.763069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.763644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.764734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.767269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.782992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 927/1820] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:38.784737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.797897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75cacc-5bc5-4c05-82f7-67117045b467 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.805027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.806135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.806914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.807974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:38.808878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:38.812092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:38.832303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 928/1821] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:38.833929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:38.838295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:38.964947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d16b3-7b8d-478c-8302-886d8748e4d0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:38.982937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:38.985622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:38.985940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:38.986467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.003925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d3a1a-c3f0-4899-86ce-e3cd9ffe71cb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:39.010227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:39.013404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:39.018231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:39.018914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:39.019194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:39.019775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.047226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:39.048413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 894/1822] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:39.051773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:39.052315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:39.074174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba31d4-71f7-45ab-a0f0-cf25181bf34f service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:39.078583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 929/1823] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:39.081124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:39.091634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80bfa0-d1f3-4e6b-a9ee-40848d60d4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:39.092322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80bfa0-d1f3-4e6b-a9ee-40848d60d4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:39.092625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80bfa0-d1f3-4e6b-a9ee-40848d60d4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:39.093057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80bfa0-d1f3-4e6b-a9ee-40848d60d4cb admin admin] PATH_INFO: `/v3/projects/bfa51fdd7c0e48839a59f8d89b78255b/users/5012a19fa4224fe299a967810843a15f/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.098524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:39.111565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79d4c-7592-415f-a8d4-23ddca772824 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:39.126573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:39.127497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:39.127836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:39.128144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.128840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:39.130556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:39.138262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80bfa0-d1f3-4e6b-a9ee-40848d60d4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa51fdd7c0e48839a59f8d89b78255b, user_id=5012a19fa4224fe299a967810843a15f, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:39.141261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80bfa0-d1f3-4e6b-a9ee-40848d60d4cb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:39.156951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 930/1824] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:39.163364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:39.171063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 895/1825] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:39 2022] PUT /identity/v3/projects/bfa51fdd7c0e48839a59f8d89b78255b/users/5012a19fa4224fe299a967810843a15f/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:39.183077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:39.183728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:39.184272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.211085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165aaf-b72c-40d1-8085-83b98b9572f6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:39.263997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] MFA Rules not processed for user `5012a19fa4224fe299a967810843a15f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:39.265404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:39.456747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:39.462202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:39.466364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:39.468559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] PATH_INFO: `/v3/projects/00d1445abf5d4e7db7a54850c683f46e/users/70fb052752964fe190792b61808c8aa5/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.521477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 896/1826] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:39.524720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:39.575644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d1445abf5d4e7db7a54850c683f46e, user_id=70fb052752964fe190792b61808c8aa5, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:39.605632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:39.681134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 931/1827] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:39 2022] PUT /identity/v3/projects/00d1445abf5d4e7db7a54850c683f46e/users/70fb052752964fe190792b61808c8aa5/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:39.689338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:39.692818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cfa5a-ea58-4405-9622-c66f67922ae1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:39.699395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:39.762231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:39.874760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:39.875548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:39.875811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:39.887765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:39.892484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:40.029633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e6e23-ba79-4f2b-a2b1-aeb985a02384 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:40.040808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:40.050211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:40.050601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:40.050888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] PATH_INFO: `/v3/projects/f6078a77309a4c30a68ec13a058d6948` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:40.071245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6078a77309a4c30a68ec13a058d6948)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:40.076751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:40.122888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:40.153891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:40.155713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:40.187207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 897/1828] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:40.189470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:40.227174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89e93-9966-4e78-8cc0-43cc47014a4e service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:40.248735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6078a77309a4c30a68ec13a058d6948', 'request_id': 'req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.274650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6078a77309a4c30a68ec13a058d6948', 'request_id': 'req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.302204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6078a77309a4c30a68ec13a058d6948', 'request_id': 'req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.345664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] The token cache is being invalidate because project f6078a77309a4c30a68ec13a058d6948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:40.350494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.353310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b4ff1132-b1d3-4a85-b9a3-eb6a953060b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.364871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 932/1829] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:39 2022] DELETE /identity/v3/projects/f6078a77309a4c30a68ec13a058d6948 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:40.366791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:40.372624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:40.372958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:40.388575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:40.389503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:40.389996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:40.390377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:40.391299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:40.393889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:40.484079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 898/1830] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:40 2022] POST /identity/v3/projects => generated 369 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:40.484079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:40.487557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] MFA Rules not processed for user `70fb052752964fe190792b61808c8aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:40.489016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:40.495340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:40.741772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 933/1831] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:40.748685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:40.750126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec82da-b9c6-4514-915d-11ff93fcfe3d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:40.754722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:40.775279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:40.777215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:40.777902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:40.780071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] PATH_INFO: `/v3/projects/0e38fb2a0d164791a7538cc9266f2cdc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:40.782745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e38fb2a0d164791a7538cc9266f2cdc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:40.791640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404cac-0ada-41d5-b4e9-f5e99fa3d85c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:40.795716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:40.803402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:40.804117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:40.804528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:40.804703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] PATH_INFO: `/v3/users/dafea32b02024e99a95dc7eb1b47457e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:40.808421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dafea32b02024e99a95dc7eb1b47457e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:40.810719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:40.892337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:40.937541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dafea32b02024e99a95dc7eb1b47457e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.957367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dafea32b02024e99a95dc7eb1b47457e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.957367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dafea32b02024e99a95dc7eb1b47457e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.958845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dafea32b02024e99a95dc7eb1b47457e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:40.985836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dafea32b02024e99a95dc7eb1b47457e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.022343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e38fb2a0d164791a7538cc9266f2cdc', 'request_id': 'req-aefdebc6-711f-46dd-bb09-34b9229e3066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.050240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 934/1832] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:40 2022] DELETE /identity/v3/users/dafea32b02024e99a95dc7eb1b47457e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:41.051388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:41.059663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:41.089879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e38fb2a0d164791a7538cc9266f2cdc', 'request_id': 'req-aefdebc6-711f-46dd-bb09-34b9229e3066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.120219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e38fb2a0d164791a7538cc9266f2cdc', 'request_id': 'req-aefdebc6-711f-46dd-bb09-34b9229e3066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.135279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:41.171044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] The token cache is being invalidate because project 0e38fb2a0d164791a7538cc9266f2cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:41.171044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.172669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.177206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 899/1833] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:40 2022] DELETE /identity/v3/projects/0e38fb2a0d164791a7538cc9266f2cdc => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:41.178144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:41.191636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:41.251281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f78549-24ee-408c-8a82-c0e101df4574 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:41.266661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:41.268834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:41.269631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:41.270223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:41.271129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:41.274115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:41.346303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 935/1834] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:41 2022] POST /identity/v3/users => generated 291 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:41.346303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:41.350728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:41.388777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780836-1de8-4279-b29f-893a2813cbe7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:41.406503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:41.409043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:41.411192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:41.412202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] PATH_INFO: `/v3/users/0ee3afccc5664960b61bfd540cc0d173` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:41.418698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee3afccc5664960b61bfd540cc0d173)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:41.424238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:41.459545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdebc6-711f-46dd-bb09-34b9229e3066 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:41.473911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:41.475007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:41.476549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:41.477190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:41.478345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:41.486778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:41.566192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee3afccc5664960b61bfd540cc0d173'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.583715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 900/1835] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:41 2022] POST /identity/v3/projects => generated 365 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:41.584019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee3afccc5664960b61bfd540cc0d173'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.585396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:41.587935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee3afccc5664960b61bfd540cc0d173'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.589256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee3afccc5664960b61bfd540cc0d173'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.590296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:41.614221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee3afccc5664960b61bfd540cc0d173'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:41.734962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 936/1836] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:41 2022] DELETE /identity/v3/users/0ee3afccc5664960b61bfd540cc0d173 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:41.738500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:41.746824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:41.798844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:41.872843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4648ea2-f825-4473-9a5a-1be7302c33e0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.115877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007077-37ff-4ce9-aeed-f888d02378f3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.126434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.127081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.127450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.127731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.128919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.130122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.131236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.131350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.131908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.132188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.133416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.141895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.190053 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 901/1837] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 605 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:42.191611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.202537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:42.207071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 937/1838] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:41 2022] POST /identity/v3/users => generated 289 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:42.209341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.249216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0776b-eb39-4c46-bda4-6fbd48b02995 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.250169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685731b8-8463-4960-871c-4743827cf7db admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.267104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.267952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.268616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.269145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.269819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.271689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.276147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.277733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.278300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.278924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] PATH_INFO: `/v3/projects/201853ea028943a68a0c098918d1b2b5/users/aa6398ce010141a295c9a3fb7624c2d8/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.313508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 938/1839] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:42.319460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.349473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201853ea028943a68a0c098918d1b2b5, user_id=aa6398ce010141a295c9a3fb7624c2d8, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.361618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.368082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242917-2ece-4cf7-9e02-0ed251a00b67 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.385348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.386617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.387670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.388709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] PATH_INFO: `/v3/projects/ee8ca583b4784aacbdf759fbac50fb12/users/2b2a154d8e5b4859815a6a2f540c8d47/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.415528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 902/1840] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:42 2022] PUT /identity/v3/projects/201853ea028943a68a0c098918d1b2b5/users/aa6398ce010141a295c9a3fb7624c2d8/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:42.418024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.444639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8ca583b4784aacbdf759fbac50fb12, user_id=2b2a154d8e5b4859815a6a2f540c8d47, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.449021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:42.456360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.509334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 939/1841] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:42 2022] PUT /identity/v3/projects/ee8ca583b4784aacbdf759fbac50fb12/users/2b2a154d8e5b4859815a6a2f540c8d47/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:42.510648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.681209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202a32e-4999-4eb4-beff-18ddbf2baab8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.688590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.690005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.690005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.690228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.691609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.693886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.713322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 940/1842] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:42.715382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.717715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb04cd3-deaa-4f2d-8fbc-f5e29615a243 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.728643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86ede7-5310-47ec-bcb8-1cffd3127714 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.736770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f43bd3-b7b5-46c6-91e9-38fceff1db79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.737285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:42.737787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f43bd3-b7b5-46c6-91e9-38fceff1db79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.738204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f43bd3-b7b5-46c6-91e9-38fceff1db79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.738477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.738686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f43bd3-b7b5-46c6-91e9-38fceff1db79 admin admin] PATH_INFO: `/v3/projects/ee8ca583b4784aacbdf759fbac50fb12/users/2b2a154d8e5b4859815a6a2f540c8d47/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.738977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.739478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.740712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.741387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f43bd3-b7b5-46c6-91e9-38fceff1db79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8ca583b4784aacbdf759fbac50fb12, user_id=2b2a154d8e5b4859815a6a2f540c8d47, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:42.747524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.761296 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f43bd3-b7b5-46c6-91e9-38fceff1db79 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:42.787915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 941/1843] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:42 2022] PUT /identity/v3/projects/ee8ca583b4784aacbdf759fbac50fb12/users/2b2a154d8e5b4859815a6a2f540c8d47/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:42.800410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 903/1844] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:42.803013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:42.803621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:42.806611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:42.812398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:42.863278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a96c3-af50-4ea0-a86c-697826940060 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:42.945683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] MFA Rules not processed for user `2b2a154d8e5b4859815a6a2f540c8d47`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:42.948566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.091597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:43.118822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d8a35-5c99-4604-a482-fb0098d57d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:43.119613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d8a35-5c99-4604-a482-fb0098d57d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.119823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d8a35-5c99-4604-a482-fb0098d57d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.120100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d8a35-5c99-4604-a482-fb0098d57d3e admin admin] PATH_INFO: `/v3/projects/201853ea028943a68a0c098918d1b2b5/users/aa6398ce010141a295c9a3fb7624c2d8/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.136127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d8a35-5c99-4604-a482-fb0098d57d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=201853ea028943a68a0c098918d1b2b5, user_id=aa6398ce010141a295c9a3fb7624c2d8, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:43.139178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d8a35-5c99-4604-a482-fb0098d57d3e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:43.192538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 904/1845] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:42 2022] PUT /identity/v3/projects/201853ea028943a68a0c098918d1b2b5/users/aa6398ce010141a295c9a3fb7624c2d8/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:43.194599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b23a6-adaa-4833-ad2a-0293ad1f00c2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.194936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b23a6-adaa-4833-ad2a-0293ad1f00c2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.195167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b23a6-adaa-4833-ad2a-0293ad1f00c2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.196891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 905/1846] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:43.209764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30c6a-6990-4533-bac8-45c00a38504e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.210103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30c6a-6990-4533-bac8-45c00a38504e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.215350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe30c6a-6990-4533-bac8-45c00a38504e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.265684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 942/1847] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:43.269297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:43.271614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.288836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bfe30c6a-6990-4533-bac8-45c00a38504e None None] MFA Rules not processed for user `aa6398ce010141a295c9a3fb7624c2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:43.290401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bfe30c6a-6990-4533-bac8-45c00a38504e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.367041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1aaed-f01c-48c3-9aea-920c6344f995 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:43.413933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 906/1848] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:43.415049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.415339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.415617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.439373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:43.440952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.458154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:43.478526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:43.479283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.479568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.479842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.576057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 907/1849] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:29:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:43.577952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:43.581492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.606724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f31a4a-2582-4c73-8032-53534edd420f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:43.613055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:43.614571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:43.618788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:43.620979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.621660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.622138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.633792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:43.636197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:43.651060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 943/1850] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:43.652633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:43.654980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.665477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 908/1851] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:43.668727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:43.671629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.695102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b61752-4aa6-43d9-ba78-abcebf5f94fb service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:43.704346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:43.705576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.706074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.706552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.720018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:43.722475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:43.740937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 909/1852] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:43.749997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:43.770368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8776e-11c4-4bbe-bdc8-43bfbb61f57a service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:43.779138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:43.781699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.782246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.782782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.783266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:43.786202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:43.879419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d0e98-cb33-4400-b149-b2299c780f06 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:43.882835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:43.884751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dc289-43d5-4f8a-9920-2a2b8ce99cf7 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:43.886341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:43.887072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:43.887308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:43.887583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] PATH_INFO: `/v3/projects/5434e721e74f4b1b8ad8705c88b03189` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:43.888954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5434e721e74f4b1b8ad8705c88b03189)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:43.892755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:43.908200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 910/1853] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:43.985781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5434e721e74f4b1b8ad8705c88b03189', 'request_id': 'req-402596ee-4662-4c09-90ca-b371749d2ed4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:43.998580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5434e721e74f4b1b8ad8705c88b03189', 'request_id': 'req-402596ee-4662-4c09-90ca-b371749d2ed4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.011837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5434e721e74f4b1b8ad8705c88b03189', 'request_id': 'req-402596ee-4662-4c09-90ca-b371749d2ed4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.029153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] The token cache is being invalidate because project 5434e721e74f4b1b8ad8705c88b03189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:44.029446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.030259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.033049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 944/1854] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:43 2022] DELETE /identity/v3/projects/5434e721e74f4b1b8ad8705c88b03189 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:44.037119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bca3ad-6616-481d-8de4-297eab0f4b3f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.037412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bca3ad-6616-481d-8de4-297eab0f4b3f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.037688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bca3ad-6616-481d-8de4-297eab0f4b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.064842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:44.067986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.084517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a3bca3ad-6616-481d-8de4-297eab0f4b3f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:44.086758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3bca3ad-6616-481d-8de4-297eab0f4b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.265383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 911/1855] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:44.276623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.281069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.281590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.314735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402596ee-4662-4c09-90ca-b371749d2ed4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:44.320971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:44.322098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.323512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.324026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] PATH_INFO: `/v3/projects/1c2a56b4d5d7410e8b512d3622b61a61` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.326242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2a56b4d5d7410e8b512d3622b61a61)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:44.328447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:44.353195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:44.356695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.395025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 912/1856] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:44.396877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:44.400548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2a56b4d5d7410e8b512d3622b61a61', 'request_id': 'req-acc4a8da-7af3-4c11-8f5f-19778091d1b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.412173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2a56b4d5d7410e8b512d3622b61a61', 'request_id': 'req-acc4a8da-7af3-4c11-8f5f-19778091d1b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.424757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2a56b4d5d7410e8b512d3622b61a61', 'request_id': 'req-acc4a8da-7af3-4c11-8f5f-19778091d1b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.428413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26113-92b0-4575-9ad0-74d4ab39432b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:44.439002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:44.469348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] The token cache is being invalidate because project 1c2a56b4d5d7410e8b512d3622b61a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:44.469348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.475663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acc4a8da-7af3-4c11-8f5f-19778091d1b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.478964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 945/1857] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:44 2022] DELETE /identity/v3/projects/1c2a56b4d5d7410e8b512d3622b61a61 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:44.491387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.492964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:44.493570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.498403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.550149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:44.551409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.609736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:44.610431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.610784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.611089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.612331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:44.612331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:44.613878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:44.649122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 913/1858] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:44.653589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:44.660528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.710316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28e31-087c-4511-92d5-989a17040d29 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:44.724440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2e65-c2a4-4fd2-b47c-2bbefecec0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:44.725117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 946/1859] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:44.725249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2e65-c2a4-4fd2-b47c-2bbefecec0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.725485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2e65-c2a4-4fd2-b47c-2bbefecec0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.725760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2e65-c2a4-4fd2-b47c-2bbefecec0ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.726622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:44.729303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b2e65-c2a4-4fd2-b47c-2bbefecec0ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:44.731328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b2e65-c2a4-4fd2-b47c-2bbefecec0ca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:44.739679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.775294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aa08e-6753-4054-a826-7871d6e811cf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:44.782973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:44.783700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.784253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.784345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] PATH_INFO: `/v3/users/70fb052752964fe190792b61808c8aa5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.788433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fb052752964fe190792b61808c8aa5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:44.795393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:44.801724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 914/1860] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:44.803681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf26a-7a8d-4a84-ae54-03ac72085ca9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.804259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf26a-7a8d-4a84-ae54-03ac72085ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.804684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf26a-7a8d-4a84-ae54-03ac72085ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:44.851346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c7daf26a-7a8d-4a84-ae54-03ac72085ca9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:44.856820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c7daf26a-7a8d-4a84-ae54-03ac72085ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.888830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fb052752964fe190792b61808c8aa5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.902823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fb052752964fe190792b61808c8aa5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.903540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fb052752964fe190792b61808c8aa5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.904162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fb052752964fe190792b61808c8aa5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.924873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fb052752964fe190792b61808c8aa5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:44.967994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 947/1861] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:44 2022] DELETE /identity/v3/users/70fb052752964fe190792b61808c8aa5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:44.968983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:44.970466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:44.980437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 915/1862] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:44.987801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:44.989043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:44.989043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.036060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:45.039790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:45.109872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bde3d6-d37b-4a84-af72-7e9673f47996 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.127967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.128722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.129021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.129294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.129986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.136736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.160577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:45.177975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 948/1863] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:45.179430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.184715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 916/1864] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:45.187226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.217336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64407fb-6804-41fa-a2c9-f32d69b1910d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.220469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7387d-31e1-444b-a1da-206ff82a1b1b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.234925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.236573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.237375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.237655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.238305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.238543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.239755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.240351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.240748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.242717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:45.243142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.245791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.248097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.279639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 949/1865] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:45.284851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.287218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:45.309026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 917/1866] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:45.309026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.330038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4376c7-40be-447c-826e-fdcf1c9c3f98 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.345791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.346844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.347333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.347934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.353220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.360517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.401694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8383b0-60bf-4655-9167-eac5ff93a9ff admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.402699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:45.426278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.427223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.427595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.427962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.436381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 918/1867] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:45 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:45.437725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.440435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.442496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.458795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 950/1868] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:45.459884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.472288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37451d-2fe8-446e-b391-c59174933e8d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.632918 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.633747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.634029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.634635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.635148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.637833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.644403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ab90a-424f-438e-8794-02297b3accbc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.653685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.655001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.655726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.656228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.657682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.660339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.675161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 951/1869] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:45 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:45.689751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 919/1870] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:45.689980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.694077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.705927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20217cac-a391-4f62-8302-2c85755c68af admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.709479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f0d3f-051a-45d8-bd99-d21be514f5a0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.716180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.717605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.718354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.719083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.719262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.720323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.721068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.722429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.722731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] PATH_INFO: `/v3/projects/bdb76655b9a3422db1a7dd40655b00d0/users/52e7788e26a146c9b5c0f631667a58b4/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.724242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.740065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 952/1871] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:45.744014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.752043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb76655b9a3422db1a7dd40655b00d0, user_id=52e7788e26a146c9b5c0f631667a58b4, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.754924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef4478-d2da-41d1-99e7-fa1fddfe4907 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.756086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.763944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eca88-4435-46d0-b61e-60c0ec04e21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.764620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eca88-4435-46d0-b61e-60c0ec04e21e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.764908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eca88-4435-46d0-b61e-60c0ec04e21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.765226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eca88-4435-46d0-b61e-60c0ec04e21e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.765906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eca88-4435-46d0-b61e-60c0ec04e21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.767785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eca88-4435-46d0-b61e-60c0ec04e21e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:45.778228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 920/1872] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:45 2022] PUT /identity/v3/projects/bdb76655b9a3422db1a7dd40655b00d0/users/52e7788e26a146c9b5c0f631667a58b4/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:45.787494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 953/1873] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:45.788746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:45.797687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.797979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.798258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.861790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] MFA Rules not processed for user `52e7788e26a146c9b5c0f631667a58b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:45.863661 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:45.932804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667731-9c8d-4a85-8545-9500cd5e9956 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:45.940669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:45.942132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:45.942715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:45.942884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] PATH_INFO: `/v3/projects/d1c5ed35cd344bf0b3430d3a83db89df/users/474bb4f0e15940f8bb3bc9183ce1e657/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:45.957663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:45.984079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c5ed35cd344bf0b3430d3a83db89df, user_id=474bb4f0e15940f8bb3bc9183ce1e657, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:45.986840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.015151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 954/1874] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:29:46.015764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.018291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.025042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 921/1875] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:45 2022] PUT /identity/v3/projects/d1c5ed35cd344bf0b3430d3a83db89df/users/474bb4f0e15940f8bb3bc9183ce1e657/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:29:46.025042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.067341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0c790-b63a-4c12-b6cf-b1cdfd9e2f2b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.130196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:46.154809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976460f5-ea0a-4d6f-b437-df82eedeb03b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.166231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b8a34-2a96-4543-88d6-0319a9c0825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.167529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b8a34-2a96-4543-88d6-0319a9c0825d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.167947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b8a34-2a96-4543-88d6-0319a9c0825d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.168344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b8a34-2a96-4543-88d6-0319a9c0825d admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.169240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b8a34-2a96-4543-88d6-0319a9c0825d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.171829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.172717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b8a34-2a96-4543-88d6-0319a9c0825d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.172891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.173312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.173776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] PATH_INFO: `/v3/projects/00d1445abf5d4e7db7a54850c683f46e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.176724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d1445abf5d4e7db7a54850c683f46e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.179292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.231511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 922/1876] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:46 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:46.231511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.231511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.231511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.268516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d1445abf5d4e7db7a54850c683f46e', 'request_id': 'req-b1e4279a-ce07-41ab-8d34-fb16b9388523'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.295439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] MFA Rules not processed for user `474bb4f0e15940f8bb3bc9183ce1e657`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:46.295876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d1445abf5d4e7db7a54850c683f46e', 'request_id': 'req-b1e4279a-ce07-41ab-8d34-fb16b9388523'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.296057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.314416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d1445abf5d4e7db7a54850c683f46e', 'request_id': 'req-b1e4279a-ce07-41ab-8d34-fb16b9388523'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.331396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] The token cache is being invalidate because project 00d1445abf5d4e7db7a54850c683f46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:46.331688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.332682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.335164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 955/1877] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:46 2022] DELETE /identity/v3/projects/00d1445abf5d4e7db7a54850c683f46e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:46.336567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.339591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.468283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 923/1878] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:46.470028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.474228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.538058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4279a-ce07-41ab-8d34-fb16b9388523 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.565184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.566318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.567035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.567523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.568730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.572923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.628154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c611a2e-8a5f-4085-ae5e-8b56e099ca4e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.635937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.636610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.636878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.637241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] PATH_INFO: `/v3/users/2b2a154d8e5b4859815a6a2f540c8d47` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.638859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2a154d8e5b4859815a6a2f540c8d47)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.641127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.660059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 956/1879] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:46 2022] POST /identity/v3/users => generated 297 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:46.660059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.668029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.681230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:46.695037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305523d-7d9b-44ac-a805-1ac4102da104 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.712106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.712889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.713260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.713677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.715086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.717807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.755515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2a154d8e5b4859815a6a2f540c8d47'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.769168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2a154d8e5b4859815a6a2f540c8d47'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.770270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2a154d8e5b4859815a6a2f540c8d47'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.771245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2a154d8e5b4859815a6a2f540c8d47'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.776366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 957/1880] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:46 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:46.777679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.780030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.788201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2a154d8e5b4859815a6a2f540c8d47'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:46.795370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:46.813969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 924/1881] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:46 2022] DELETE /identity/v3/users/2b2a154d8e5b4859815a6a2f540c8d47 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:46.815431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.818734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:46.867875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbaf76-7653-4ccb-86fb-82b6bd3d21f4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.930648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052580e-d3f7-4746-9589-d65211bdd5e8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.936246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.936936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.937283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.937594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.938397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.940663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.967354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 925/1882] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:46 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:46.971701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:46.974036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.975556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.976191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.977107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.979701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.982182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:46.985140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d318fd0-c191-427a-94bf-d387838ada18 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:46.990861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:46.991636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:46.992018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:46.992413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:46.993299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:46.995316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.000882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 958/1883] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:47.008540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.023089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 926/1884] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:29:47.025047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.028958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd8d5-0f06-48f4-99c8-d15f6ea1c680 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.037466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.038258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.038612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.038954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] PATH_INFO: `/v3/projects/79d77a170f9242b2af0c5fd57bbee2c2/users/73d0382c7a064eb4b317107ecee29694/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.044559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e11b4-7e1f-403f-97a9-6a560993b816 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.050691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.051382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.052567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.052567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] PATH_INFO: `/v3/projects/b5d3a4c84c334f0e9d47f6f0436ac5ad/users/eda09e8a2f694aebb145eae9b2363dae/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.060479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d77a170f9242b2af0c5fd57bbee2c2, user_id=73d0382c7a064eb4b317107ecee29694, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.063189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.083419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 959/1885] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:47 2022] PUT /identity/v3/projects/79d77a170f9242b2af0c5fd57bbee2c2/users/73d0382c7a064eb4b317107ecee29694/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:47.083756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d3a4c84c334f0e9d47f6f0436ac5ad, user_id=eda09e8a2f694aebb145eae9b2363dae, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.085236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.087685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.113223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 927/1886] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:47 2022] PUT /identity/v3/projects/b5d3a4c84c334f0e9d47f6f0436ac5ad/users/eda09e8a2f694aebb145eae9b2363dae/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:29:47.119383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.208678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5046bf-40f0-4927-bcc4-84673e2f1b5e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.222843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:47.260235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a7404-dc75-490f-878d-666991d5cef8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.265623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.266479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.266954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.267323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.268081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.269923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.282684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 928/1887] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:47.283478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.285671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.306411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c1df-6ddf-4f91-ae4f-6bff8ff255fd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.315403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.316188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.316469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.316803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] PATH_INFO: `/v3/projects/ee8ca583b4784aacbdf759fbac50fb12` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.318591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8ca583b4784aacbdf759fbac50fb12)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.321629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.355742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.356575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.356953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.357839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.358762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.361063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.377886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:47.378476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 960/1888] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:29:47.379505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.396168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abe3c1-330a-4125-a9d8-cf2a84db4077 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.404158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8ca583b4784aacbdf759fbac50fb12', 'request_id': 'req-fb21f08b-bedf-4dbb-b2cb-e1832508382c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:47.412601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8ca583b4784aacbdf759fbac50fb12', 'request_id': 'req-fb21f08b-bedf-4dbb-b2cb-e1832508382c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:47.423349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8ca583b4784aacbdf759fbac50fb12', 'request_id': 'req-fb21f08b-bedf-4dbb-b2cb-e1832508382c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:47.438719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] The token cache is being invalidate because project ee8ca583b4784aacbdf759fbac50fb12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:47.439045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:47.439999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:47.441879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 929/1889] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:47 2022] DELETE /identity/v3/projects/ee8ca583b4784aacbdf759fbac50fb12 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:47.442562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.444212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.491262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa19e3-9bc1-4c2c-b6ff-a883304ab85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.492410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa19e3-9bc1-4c2c-b6ff-a883304ab85f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.492810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa19e3-9bc1-4c2c-b6ff-a883304ab85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.493709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa19e3-9bc1-4c2c-b6ff-a883304ab85f admin admin] PATH_INFO: `/v3/projects/b5d3a4c84c334f0e9d47f6f0436ac5ad/users/eda09e8a2f694aebb145eae9b2363dae/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.497192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa19e3-9bc1-4c2c-b6ff-a883304ab85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d3a4c84c334f0e9d47f6f0436ac5ad, user_id=eda09e8a2f694aebb145eae9b2363dae, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.502123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa19e3-9bc1-4c2c-b6ff-a883304ab85f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.529807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 961/1890] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:47 2022] PUT /identity/v3/projects/b5d3a4c84c334f0e9d47f6f0436ac5ad/users/eda09e8a2f694aebb145eae9b2363dae/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:47.529807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed407d-223a-49b5-8a33-50cf6b4ce008 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.529807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed407d-223a-49b5-8a33-50cf6b4ce008 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.529807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed407d-223a-49b5-8a33-50cf6b4ce008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.554368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21f08b-bedf-4dbb-b2cb-e1832508382c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:47.556536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d1ed407d-223a-49b5-8a33-50cf6b4ce008 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:47.558787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1ed407d-223a-49b5-8a33-50cf6b4ce008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.648844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 962/1891] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:47.650927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171c8fb-ffe7-41b9-bc66-93d8cffc86b7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.651213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171c8fb-ffe7-41b9-bc66-93d8cffc86b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.651489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171c8fb-ffe7-41b9-bc66-93d8cffc86b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.661699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b207a75-6230-43f7-9a8e-0461f767411c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:47.662387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b207a75-6230-43f7-9a8e-0461f767411c admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.662665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b207a75-6230-43f7-9a8e-0461f767411c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.663081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b207a75-6230-43f7-9a8e-0461f767411c admin admin] PATH_INFO: `/v3/projects/79d77a170f9242b2af0c5fd57bbee2c2/users/73d0382c7a064eb4b317107ecee29694/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.665105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b207a75-6230-43f7-9a8e-0461f767411c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d77a170f9242b2af0c5fd57bbee2c2, user_id=73d0382c7a064eb4b317107ecee29694, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:47.667890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b207a75-6230-43f7-9a8e-0461f767411c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:47.689039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 930/1892] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:47 2022] PUT /identity/v3/projects/79d77a170f9242b2af0c5fd57bbee2c2/users/73d0382c7a064eb4b317107ecee29694/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:47.689039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be695e-600a-4075-9566-451e57ec7094 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.689039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be695e-600a-4075-9566-451e57ec7094 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.689039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be695e-600a-4075-9566-451e57ec7094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.695160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6171c8fb-ffe7-41b9-bc66-93d8cffc86b7 None None] MFA Rules not processed for user `eda09e8a2f694aebb145eae9b2363dae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:47.695160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6171c8fb-ffe7-41b9-bc66-93d8cffc86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.719179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-36be695e-600a-4075-9566-451e57ec7094 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:47.720868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-36be695e-600a-4075-9566-451e57ec7094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.732210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:47.814472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 963/1893] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:47.815891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:47.816178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:47.816451 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:47.883074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 931/1894] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:47.883074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be695e-600a-4075-9566-451e57ec7094 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:47.887391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] MFA Rules not processed for user `73d0382c7a064eb4b317107ecee29694`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:47.890511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.893709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-36be695e-600a-4075-9566-451e57ec7094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:47.914618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:48.036551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 964/1895] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:48.038087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.060327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa957811-d3a2-436d-ae2e-2045caccb4e8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.071231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.072071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.072498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.072872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.073986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:48.074412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.076446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.079023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be695e-600a-4075-9566-451e57ec7094 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.085914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.087186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.087379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.087849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.097173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 965/1896] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:48.097173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.100740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:48.113604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.116877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.118063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896403e-29ae-4a5c-8667-becaa67b8943 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.131180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.132745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.133652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 932/1897] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:48.134310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.134540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.135583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.155940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.158020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.165721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338935cc-c27b-4a4d-a677-45d9cb6fc17a service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.173806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.174859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.175476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.176008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.176920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.179140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.197330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 966/1898] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:48.284943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 933/1899] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:48 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:48.297795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.358540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3666b51-5fd7-4f19-b8d6-7b0135144461 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.450300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.522674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.523613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.524234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.524846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.526773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.531257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.582371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 967/1900] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:48 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:48.599165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.632777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91e2d-bb56-4c5f-88a1-88d95af11b7f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.644190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.645829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.646951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.647604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.649283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.654174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.666947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a9f55-fb48-4c86-8e6f-513fc89846b9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.678903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.680286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.681677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.682224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.689521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 968/1901] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:48.691487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:48.693565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.715061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:48.754790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4d7c9-81a9-498e-988f-c2a490595ce1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.774391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.776365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.776852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.777554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.781310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.786183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.830970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 969/1902] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:48.830970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.835071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:48.854103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.855619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.889388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 934/1903] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:48.889575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ffcc2-0d3f-4ebe-bd80-f71375b1f19c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.898808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:48.910688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.911771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.912260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.912743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] PATH_INFO: `/v3/users/474bb4f0e15940f8bb3bc9183ce1e657` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.917018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=474bb4f0e15940f8bb3bc9183ce1e657)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:48.925608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:48.943041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7450a0-ce25-4919-88c9-58f4716e20fe service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:48.957425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d555109-b6e2-4135-b45c-1a98e1320931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:48.958117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555109-b6e2-4135-b45c-1a98e1320931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:48.958401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555109-b6e2-4135-b45c-1a98e1320931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:48.958679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d555109-b6e2-4135-b45c-1a98e1320931 admin admin] PATH_INFO: `/v3/projects/fc4c7a06b8504311a5664364f41a31b7/users/bc062f2c0d8e42748ee73cd41707745e/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:48.999074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d555109-b6e2-4135-b45c-1a98e1320931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4c7a06b8504311a5664364f41a31b7, user_id=bc062f2c0d8e42748ee73cd41707745e, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:49.011736 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d555109-b6e2-4135-b45c-1a98e1320931 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:49.063165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 935/1904] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:48 2022] PUT /identity/v3/projects/fc4c7a06b8504311a5664364f41a31b7/users/bc062f2c0d8e42748ee73cd41707745e/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:49.088483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a62f-0d10-40e7-a654-2e6242d67876 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:49.088951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a62f-0d10-40e7-a654-2e6242d67876 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:49.089464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a62f-0d10-40e7-a654-2e6242d67876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:49.118800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474bb4f0e15940f8bb3bc9183ce1e657'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.154138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474bb4f0e15940f8bb3bc9183ce1e657'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.154138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474bb4f0e15940f8bb3bc9183ce1e657'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.154138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474bb4f0e15940f8bb3bc9183ce1e657'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.167877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:49.175595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6de2a62f-0d10-40e7-a654-2e6242d67876 None None] MFA Rules not processed for user `bc062f2c0d8e42748ee73cd41707745e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:49.179135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6de2a62f-0d10-40e7-a654-2e6242d67876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:49.191981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474bb4f0e15940f8bb3bc9183ce1e657'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.251962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 970/1905] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:48 2022] DELETE /identity/v3/users/474bb4f0e15940f8bb3bc9183ce1e657 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:49.253555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:49.256042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:49.392437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19637512-bcaa-47cd-8ab2-43c5f8121bfb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:49.392437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:49.392437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:49.392437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:49.393275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] PATH_INFO: `/v3/users/5012a19fa4224fe299a967810843a15f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:49.393275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5012a19fa4224fe299a967810843a15f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:49.395419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:49.470777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 936/1906] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:29:49.474495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:49.474926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:49.475336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:49.520619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:49.522938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:49.544592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 937/1907] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:49.552550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:49.555843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:49.569048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5012a19fa4224fe299a967810843a15f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.591132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a7cf4-b232-41e8-a3d6-2e66ca65a357 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:49.617035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c3524-3044-42e4-b018-70c84a8d1bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:49.618273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c3524-3044-42e4-b018-70c84a8d1bcb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:49.618826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c3524-3044-42e4-b018-70c84a8d1bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:49.620052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c3524-3044-42e4-b018-70c84a8d1bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:49.621488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c3524-3044-42e4-b018-70c84a8d1bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:49.623487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5012a19fa4224fe299a967810843a15f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.625205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c3524-3044-42e4-b018-70c84a8d1bcb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:49.625449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5012a19fa4224fe299a967810843a15f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.631122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5012a19fa4224fe299a967810843a15f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.658371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5012a19fa4224fe299a967810843a15f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:49.663452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:49.710587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 971/1908] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:49 2022] DELETE /identity/v3/users/5012a19fa4224fe299a967810843a15f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:49.712595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:49.716132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:49.739352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 938/1909] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:49 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:49.739352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:49.739352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:49.739352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:49.856016 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:49.858446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:49.879391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4561e9-a279-4348-a2e3-4aeb29d3274a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:49.993485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:49.994705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:49.995296 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:49.995824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:49.999992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:50.007310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 939/1910] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:50.007310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.007310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:50.028627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.030536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.033908 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9b330-ee84-4bb8-ad5e-460d42984bb7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:50.045596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 972/1911] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:50.046681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:50.047185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.047947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.048522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.049035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:50.049205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.050518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.054835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.061955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937e6f-76b1-4b2f-88db-302d27108cf3 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:50.067515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:50.068216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.068657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.069036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.087473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.089830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.103300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 973/1912] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:50.111116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 940/1913] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:50 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:50.114379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.126759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407372e3-7a7a-412c-a8a3-2518635b7b54 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:50.134978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:50.135934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.136307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.136908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.137935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.140227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.158207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 974/1914] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:50.162526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.186645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed429c-9181-4213-851d-ee91034db966 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:50.200636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a69b5-09f8-4f4c-a05e-dc8dd1da8ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:50.202100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a69b5-09f8-4f4c-a05e-dc8dd1da8ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.202818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a69b5-09f8-4f4c-a05e-dc8dd1da8ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.207853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a69b5-09f8-4f4c-a05e-dc8dd1da8ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.207853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a69b5-09f8-4f4c-a05e-dc8dd1da8ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.208946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a69b5-09f8-4f4c-a05e-dc8dd1da8ff0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.241238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 941/1915] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:50.243180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.259788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7166f-6827-4fbf-9b0b-d8dc3c9f80b1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:50.271473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:50.272171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.272459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.275256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] PATH_INFO: `/v3/projects/77bf920f924f424795abd93b421afa69/users/91de82877474402abbc10dc220c4fbb1/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.310072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bf920f924f424795abd93b421afa69, user_id=91de82877474402abbc10dc220c4fbb1, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.314391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.333125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 975/1916] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:50 2022] PUT /identity/v3/projects/77bf920f924f424795abd93b421afa69/users/91de82877474402abbc10dc220c4fbb1/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:50.352798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.354110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.354698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.459456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] MFA Rules not processed for user `91de82877474402abbc10dc220c4fbb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:50.462575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:50.531613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:50.613292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 942/1917] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:50.620593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.766356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:50.802359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a9631-0553-45f5-bbbe-5fb6e5201212 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:50.814066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:50.817388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:50.818187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:50.819785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:50.821785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:50.827491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:50.900123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 976/1918] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:50 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:50.907529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:50.937206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff9fbc-267a-491d-b584-ab0a00a0f751 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:51.116739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:51.122966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:51.199367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57e763-ac01-480d-8f47-efcebd2cd8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:51.203558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57e763-ac01-480d-8f47-efcebd2cd8fd admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:51.204160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57e763-ac01-480d-8f47-efcebd2cd8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:51.205139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57e763-ac01-480d-8f47-efcebd2cd8fd admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:51.206735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57e763-ac01-480d-8f47-efcebd2cd8fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:51.216025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57e763-ac01-480d-8f47-efcebd2cd8fd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:51.344880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 943/1919] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:50 2022] POST /identity/v3/users => generated 300 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:51.354060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04db181-d832-4cc9-b3f6-6caa484dcd5b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:51.355428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04db181-d832-4cc9-b3f6-6caa484dcd5b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:51.358212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04db181-d832-4cc9-b3f6-6caa484dcd5b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:51.369146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 944/1920] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:29:51 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:29:51.384916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04db181-d832-4cc9-b3f6-6caa484dcd5b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:51.474456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04db181-d832-4cc9-b3f6-6caa484dcd5b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:51.483120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af66c8e-b41c-43fd-abea-8e019006a7f8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:51.501875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:51.502960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:51.503374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:51.503529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8690e2b-4608-4f04-bfc6-55fad5ac5849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:51.503854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] PATH_INFO: `/v3/projects/bfa51fdd7c0e48839a59f8d89b78255b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:51.505055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690e2b-4608-4f04-bfc6-55fad5ac5849 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:51.505055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690e2b-4608-4f04-bfc6-55fad5ac5849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:51.505366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8690e2b-4608-4f04-bfc6-55fad5ac5849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:51.506851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8690e2b-4608-4f04-bfc6-55fad5ac5849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:51.514446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa51fdd7c0e48839a59f8d89b78255b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:51.518042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:51.518290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8690e2b-4608-4f04-bfc6-55fad5ac5849 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:51.569583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 945/1921] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:51.574626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:51.575226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:51.575729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:51.691171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:51.693777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:51.826408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa51fdd7c0e48839a59f8d89b78255b', 'request_id': 'req-568b93e6-4199-49a7-803c-3a7db7f2650c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:51.846853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa51fdd7c0e48839a59f8d89b78255b', 'request_id': 'req-568b93e6-4199-49a7-803c-3a7db7f2650c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:51.881863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa51fdd7c0e48839a59f8d89b78255b', 'request_id': 'req-568b93e6-4199-49a7-803c-3a7db7f2650c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:51.959509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] The token cache is being invalidate because project bfa51fdd7c0e48839a59f8d89b78255b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:51.962738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:51.979205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:51.998155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 946/1923] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:52.000944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:52.005071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 977/1923] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:51 2022] DELETE /identity/v3/projects/bfa51fdd7c0e48839a59f8d89b78255b => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:29:52.010853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:52.019969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:52.033436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbf330-3013-4c21-8762-81003ef910e7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:52.168296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:52.216929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:52.217954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:52.218425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:52.220561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] PATH_INFO: `/v3/projects/f1161df8fd6b482b8f937107721ba4e1/users/9edca9ea031748a1bbae78de7b4336fb/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:52.266274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b93e6-4199-49a7-803c-3a7db7f2650c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:52.280329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1161df8fd6b482b8f937107721ba4e1, user_id=9edca9ea031748a1bbae78de7b4336fb, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:52.283785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:52.289670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:52.290390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:52.290671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:52.291195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] PATH_INFO: `/v3/users/aa6398ce010141a295c9a3fb7624c2d8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:52.293255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6398ce010141a295c9a3fb7624c2d8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:52.296173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:52.308356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 947/1924] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:51 2022] PUT /identity/v3/projects/f1161df8fd6b482b8f937107721ba4e1/users/9edca9ea031748a1bbae78de7b4336fb/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:52.310882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:52.312924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:52.315239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:52.395037 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6398ce010141a295c9a3fb7624c2d8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.413042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6398ce010141a295c9a3fb7624c2d8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.414549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6398ce010141a295c9a3fb7624c2d8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.415572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6398ce010141a295c9a3fb7624c2d8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.432757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6398ce010141a295c9a3fb7624c2d8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.465566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e690b-8219-4f63-852c-6221d06b6212 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:52.481911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 978/1925] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:52 2022] DELETE /identity/v3/users/aa6398ce010141a295c9a3fb7624c2d8 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:52.484104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:52.486826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:52.502180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:52.503430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:52.505177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:52.507020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:52.516478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:52.571661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:52.585036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:52.612923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 948/1926] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:52.616374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:52.624484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:52.678258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a548b2-1e74-429e-9a13-56e206b60a6d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:52.687248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:52.688085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:52.688225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:52.688527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] PATH_INFO: `/v3/projects/d1c5ed35cd344bf0b3430d3a83db89df` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:52.689942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c5ed35cd344bf0b3430d3a83db89df)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:52.706405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:52.933513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c5ed35cd344bf0b3430d3a83db89df', 'request_id': 'req-d8138e06-568a-4f96-a4c6-0e0632f0d951'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.957463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7d2de-5f62-4bbe-96dc-2aa3c77dbfa3 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:52.960003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c5ed35cd344bf0b3430d3a83db89df', 'request_id': 'req-d8138e06-568a-4f96-a4c6-0e0632f0d951'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:52.990962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c5ed35cd344bf0b3430d3a83db89df', 'request_id': 'req-d8138e06-568a-4f96-a4c6-0e0632f0d951'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.021227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] The token cache is being invalidate because project d1c5ed35cd344bf0b3430d3a83db89df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:53.021517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.022282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.028492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 979/1927] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:52 2022] DELETE /identity/v3/projects/d1c5ed35cd344bf0b3430d3a83db89df => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:53.032075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:53.036320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:53.086766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:53.137227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:53.137948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:53.138344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:53.138683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:53.139653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:53.141600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:53.145202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:53.182509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 949/1928] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 569 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:53.184729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:53.191111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:53.238382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb95ca-684a-46db-b141-e0de46426069 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:53.263707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:53.267356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:53.268037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:53.268604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] PATH_INFO: `/v3/users/eda09e8a2f694aebb145eae9b2363dae` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:53.275994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda09e8a2f694aebb145eae9b2363dae)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:53.280438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:53.301952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8138e06-568a-4f96-a4c6-0e0632f0d951 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:53.324798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:53.326338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:53.326640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:53.327228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] PATH_INFO: `/v3/users/52e7788e26a146c9b5c0f631667a58b4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:53.334175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e7788e26a146c9b5c0f631667a58b4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:53.339297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:53.455732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda09e8a2f694aebb145eae9b2363dae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.482986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda09e8a2f694aebb145eae9b2363dae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.484112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda09e8a2f694aebb145eae9b2363dae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.487958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda09e8a2f694aebb145eae9b2363dae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.506237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda09e8a2f694aebb145eae9b2363dae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.508093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e7788e26a146c9b5c0f631667a58b4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.538633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e7788e26a146c9b5c0f631667a58b4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.543385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e7788e26a146c9b5c0f631667a58b4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.547351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e7788e26a146c9b5c0f631667a58b4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.563361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 950/1929] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:53 2022] DELETE /identity/v3/users/eda09e8a2f694aebb145eae9b2363dae => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:53.563361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:53.566049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:53.571444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e7788e26a146c9b5c0f631667a58b4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:53.659321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 980/1930] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:53 2022] DELETE /identity/v3/users/52e7788e26a146c9b5c0f631667a58b4 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:53.660528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:53.666518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:53.810971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e533c-5f59-4a81-9dfa-963d686f3f4c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:53.854728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb9afe-2afd-4c6f-a128-1feeb65ff3e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:53.868459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:53.869386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:53.869671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:53.869950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] PATH_INFO: `/v3/projects/201853ea028943a68a0c098918d1b2b5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:53.873033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201853ea028943a68a0c098918d1b2b5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:53.877221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:53.942930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:53.944120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:53.944804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:53.945278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] PATH_INFO: `/v3/projects/f1161df8fd6b482b8f937107721ba4e1/users/9edca9ea031748a1bbae78de7b4336fb/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:53.949536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1161df8fd6b482b8f937107721ba4e1, user_id=9edca9ea031748a1bbae78de7b4336fb, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:53.953418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:53.970202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 951/1931] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:53 2022] PUT /identity/v3/projects/f1161df8fd6b482b8f937107721ba4e1/users/9edca9ea031748a1bbae78de7b4336fb/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:53.971440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:53.977147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.006178 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:54.010122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:54.036264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201853ea028943a68a0c098918d1b2b5', 'request_id': 'req-9cbedea0-645e-4338-8a9d-1ae8440a4558'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.064980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201853ea028943a68a0c098918d1b2b5', 'request_id': 'req-9cbedea0-645e-4338-8a9d-1ae8440a4558'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.085631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201853ea028943a68a0c098918d1b2b5', 'request_id': 'req-9cbedea0-645e-4338-8a9d-1ae8440a4558'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.119617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] The token cache is being invalidate because project 201853ea028943a68a0c098918d1b2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:54.120281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.121697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9cbedea0-645e-4338-8a9d-1ae8440a4558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.124770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 981/1932] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:53 2022] DELETE /identity/v3/projects/201853ea028943a68a0c098918d1b2b5 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:54.127589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93196cf-50d9-44b0-88d1-19e80808a087 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:54.128374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93196cf-50d9-44b0-88d1-19e80808a087 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:54.128952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93196cf-50d9-44b0-88d1-19e80808a087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:54.176246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e93196cf-50d9-44b0-88d1-19e80808a087 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:54.180012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e93196cf-50d9-44b0-88d1-19e80808a087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.209465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd05e4-9cf2-482d-ae0b-e7acf3f7e595 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:54.325145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 982/1933] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:54.326708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419357a-5d01-45f1-a6e4-d82de962841e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:54.327062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419357a-5d01-45f1-a6e4-d82de962841e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:54.327685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419357a-5d01-45f1-a6e4-d82de962841e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:54.371758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1419357a-5d01-45f1-a6e4-d82de962841e None None] MFA Rules not processed for user `9edca9ea031748a1bbae78de7b4336fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:54.373030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1419357a-5d01-45f1-a6e4-d82de962841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.376003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:54.377437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:54.378032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:54.378608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] PATH_INFO: `/v3/users/3e0c3322cc6e494db65980cf309f3e0a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:54.381157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e0c3322cc6e494db65980cf309f3e0a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:54.384336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:54.516127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e0c3322cc6e494db65980cf309f3e0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.531990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 983/1934] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:54.531990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:54.531990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:54.531990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:54.536840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e0c3322cc6e494db65980cf309f3e0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.538041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e0c3322cc6e494db65980cf309f3e0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.539104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e0c3322cc6e494db65980cf309f3e0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.561183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:54.563208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.574381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 984/1935] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:54.575540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:54.583104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.587931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e0c3322cc6e494db65980cf309f3e0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:54.659435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:54.668276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 952/1936] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:53 2022] DELETE /identity/v3/users/3e0c3322cc6e494db65980cf309f3e0a => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:54.668477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ece79-65fd-4893-997c-6dacf782a195 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:54.668560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:54.673054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.770846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:54.773534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:54.775055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:54.776232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:54.776703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:54.782971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.820210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8064bd6-9a7b-44d5-972b-79b75ffa9811 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:54.827110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:54.828317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:54.828960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:54.829602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:54.835068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:54.987520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:54.999243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e16fe-7170-4cd1-b5db-832a536870da service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.039640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 985/1937] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:55.041719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.042121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.042278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.044322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.048899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84fe3-e897-4178-ba35-b52468555d1f service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.065964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:55.067254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.078448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 953/1938] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:55.080593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.080878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.081183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.085694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 986/1939] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:55.086689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.090482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.112048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cf3c4-74b0-4ff0-92fa-e04aef5f5510 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.123630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:55.124895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.135267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.136823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.136964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.137823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 954/1940] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:55.138824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.143948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] PATH_INFO: `/v3/projects/b5d3a4c84c334f0e9d47f6f0436ac5ad` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.147446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d3a4c84c334f0e9d47f6f0436ac5ad)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.151093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158e056-42f3-4573-9022-ff5d63281aa1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.158087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.158874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.159377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.159664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.160535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:55.160850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.162979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.169411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.179620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 955/1941] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:55.180967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.185356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.210349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49ed3-32c5-4be8-aa57-c19858971c8c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.226518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.226978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.227259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.227532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.268049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.269582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.296698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 956/1942] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:55.298169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.311197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c673a75-c761-4eca-9bbe-b4d44e26177f service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.325524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.326221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.326472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.326750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.327412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.329582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.378197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 957/1943] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:55 2022] POST /identity/v3/projects => generated 399 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:55.383429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.385552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d3a4c84c334f0e9d47f6f0436ac5ad', 'request_id': 'req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.402583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d3a4c84c334f0e9d47f6f0436ac5ad', 'request_id': 'req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.420911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d3a4c84c334f0e9d47f6f0436ac5ad', 'request_id': 'req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.439637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] The token cache is being invalidate because project b5d3a4c84c334f0e9d47f6f0436ac5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:55.440015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.441674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-514665fa-f2dc-46eb-98cd-94b31b2e8fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.446755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 987/1944] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:55 2022] DELETE /identity/v3/projects/b5d3a4c84c334f0e9d47f6f0436ac5ad => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:55.450696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.450696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.451866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.484581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:55.487289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.531706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84eab-269c-4ae0-b829-38941f0c58f7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.546900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.548441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.548441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.548441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.549246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.551855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.559929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:55.596248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 958/1945] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:55 2022] POST /identity/v3/users => generated 300 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:55.598033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.601706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.624433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5433d1-09dd-403f-8129-1bb06d1cc932 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.636006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.637045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.637671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.637961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] PATH_INFO: `/v3/projects/bdb76655b9a3422db1a7dd40655b00d0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.640323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb76655b9a3422db1a7dd40655b00d0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.643202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.650638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 988/1946] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:55.651731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.658583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.684891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c711e6-dc6c-4073-8a2c-c3cd9f268757 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.685685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:55.696570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885cf27-94ac-4b14-b760-f7ef6d5cd567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.698257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885cf27-94ac-4b14-b760-f7ef6d5cd567 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.699670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885cf27-94ac-4b14-b760-f7ef6d5cd567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.699999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885cf27-94ac-4b14-b760-f7ef6d5cd567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.700958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885cf27-94ac-4b14-b760-f7ef6d5cd567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:55.705313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885cf27-94ac-4b14-b760-f7ef6d5cd567 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:55.731976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 989/1947] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:55.733584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.733872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.735010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.770758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb76655b9a3422db1a7dd40655b00d0', 'request_id': 'req-51be3703-3afe-46e2-b23b-14dfc999f600'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.786235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:55.789678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.793883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb76655b9a3422db1a7dd40655b00d0', 'request_id': 'req-51be3703-3afe-46e2-b23b-14dfc999f600'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.804134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb76655b9a3422db1a7dd40655b00d0', 'request_id': 'req-51be3703-3afe-46e2-b23b-14dfc999f600'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.823459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] The token cache is being invalidate because project bdb76655b9a3422db1a7dd40655b00d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:55.823953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.824972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:55.830259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 959/1948] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:55 2022] DELETE /identity/v3/projects/bdb76655b9a3422db1a7dd40655b00d0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:55.831260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.836415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.922199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 990/1949] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:55.923248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:55.924908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:55.983242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b70ae-375e-4c30-8073-af35fcd60a24 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:55.991496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:55.992405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:55.992906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:55.993371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] PATH_INFO: `/v3/users/73d0382c7a064eb4b317107ecee29694` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:55.996076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d0382c7a064eb4b317107ecee29694)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.001814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.048763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be3703-3afe-46e2-b23b-14dfc999f600 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.057269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.057269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.057269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.057269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.058197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.062304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.084477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 960/1950] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:56.085682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.092203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d0382c7a064eb4b317107ecee29694'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:56.105665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b559-a47c-44a8-a449-809187d2bb91 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.105917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d0382c7a064eb4b317107ecee29694'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:56.106523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d0382c7a064eb4b317107ecee29694'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:56.107230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d0382c7a064eb4b317107ecee29694'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:56.119704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.120878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.122059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.122317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d0382c7a064eb4b317107ecee29694'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:56.122406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.123620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:56.125385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.129660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.148872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 991/1951] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:55 2022] DELETE /identity/v3/users/73d0382c7a064eb4b317107ecee29694 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:56.149469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.154742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:56.174104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 961/1952] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:56.180467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.184591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:56.209231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:56.226780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f4af-125b-4fbe-bfb9-6a4eaf17fe39 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.271247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc4444-1a60-4d1a-86cd-a875d356eeff admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.279642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.280474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.281287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.281634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] PATH_INFO: `/v3/projects/36c2cfc1060040ffa358a31a42e83af4/users/2303f076041b471cbb79dd094c4bf476/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.338619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c2cfc1060040ffa358a31a42e83af4, user_id=2303f076041b471cbb79dd094c4bf476, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.351424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.385986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d03a42-8044-46c7-b576-8ae217e549c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.386671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d03a42-8044-46c7-b576-8ae217e549c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.386979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d03a42-8044-46c7-b576-8ae217e549c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.387256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d03a42-8044-46c7-b576-8ae217e549c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.387912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d03a42-8044-46c7-b576-8ae217e549c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.389965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d03a42-8044-46c7-b576-8ae217e549c2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.393933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 992/1953] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:56 2022] PUT /identity/v3/projects/36c2cfc1060040ffa358a31a42e83af4/users/2303f076041b471cbb79dd094c4bf476/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:56.395771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.397924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:56.443857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 962/1954] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:56 2022] POST /identity/v3/projects => generated 407 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:56.445789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.446200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.447438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.486627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:56.492822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] MFA Rules not processed for user `2303f076041b471cbb79dd094c4bf476`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:56.494753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:56.551611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51eb35-3827-4024-8d7f-00b63a3c71ea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.627620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 963/1955] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:56.628841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.699774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.700201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.701971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.702372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.707130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:56.779659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58249a11-daaa-45b8-92ec-d03f70d172a2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.790324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.792626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.792963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.793282 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.794786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.797267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.851007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.854465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.864466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 964/1956] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:56 2022] POST /identity/v3/users => generated 304 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:56.867727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.871201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:56.885080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 993/1957] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:56.887474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:56.898720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a53c2-1a19-4195-b3cb-a30cd58713ea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.906279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.906641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.907050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.907419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] PATH_INFO: `/v3/projects/f4cd775156864b8689cd8b4fe864e2be` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.909223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4cd775156864b8689cd8b4fe864e2be)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.911732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:56.918500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4c9d8-1ad7-47d5-aedc-00b28af7f6dd service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:56.932690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:56.933867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:56.934295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:56.934711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:56.935546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:56.938226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.003174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 994/1958] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:56 2022] POST /identity/v3/projects => generated 401 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:57.009238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:57.014609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:57.018466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4cd775156864b8689cd8b4fe864e2be', 'request_id': 'req-536299b6-ed33-4c6c-9ea1-8f17539c9633'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.062160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4cd775156864b8689cd8b4fe864e2be', 'request_id': 'req-536299b6-ed33-4c6c-9ea1-8f17539c9633'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.082849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4cd775156864b8689cd8b4fe864e2be', 'request_id': 'req-536299b6-ed33-4c6c-9ea1-8f17539c9633'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.108110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] The token cache is being invalidate because project f4cd775156864b8689cd8b4fe864e2be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:57.108492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.113995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.123855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 965/1959] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:56 2022] DELETE /identity/v3/projects/f4cd775156864b8689cd8b4fe864e2be => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:57.126142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:57.130619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:57.180760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990c79d-41a0-446e-baa8-267dc8acc657 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:57.201092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:57.201594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:57.201773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:57.202063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:57.202344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:57.203031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:57.209682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.261876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 995/1960] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:57.263741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:57.267915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:57.317109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91db20c-c9d9-4ff4-9acf-77915940d165 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:57.334956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536299b6-ed33-4c6c-9ea1-8f17539c9633 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:57.335730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:57.338104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:57.339447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:57.340493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:57.341276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:57.346667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.348776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:57.350335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:57.351042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:57.351362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] PATH_INFO: `/v3/projects/79d77a170f9242b2af0c5fd57bbee2c2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:57.358436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d77a170f9242b2af0c5fd57bbee2c2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:57.362641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.432708 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 996/1961] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:57 2022] POST /identity/v3/users => generated 307 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:57.435425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:57.451802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:57.499684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ae547-3354-4452-908a-906be68a6b81 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:57.550383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d77a170f9242b2af0c5fd57bbee2c2', 'request_id': 'req-22e508f2-f671-41d6-8801-4dfa8b679f02'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.565147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d77a170f9242b2af0c5fd57bbee2c2', 'request_id': 'req-22e508f2-f671-41d6-8801-4dfa8b679f02'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.634448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d77a170f9242b2af0c5fd57bbee2c2', 'request_id': 'req-22e508f2-f671-41d6-8801-4dfa8b679f02'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.714283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] The token cache is being invalidate because project 79d77a170f9242b2af0c5fd57bbee2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:57.719311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.723794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.740326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 966/1962] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:57 2022] DELETE /identity/v3/projects/79d77a170f9242b2af0c5fd57bbee2c2 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:57.749603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:57.758060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:57.778750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:57.779434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:57.779896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:57.780083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] PATH_INFO: `/v3/users/1bdd227aff83453bac3abf9dfb291b30` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:57.784437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdd227aff83453bac3abf9dfb291b30)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:57.788271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.828642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e508f2-f671-41d6-8801-4dfa8b679f02 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:57.848566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:57.849236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:57.849510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:57.849789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:57.850456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:57.854777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.921538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 967/1963] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:57.922422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:57.930180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:57.933334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdd227aff83453bac3abf9dfb291b30'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.953334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdd227aff83453bac3abf9dfb291b30'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.954012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdd227aff83453bac3abf9dfb291b30'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.955126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdd227aff83453bac3abf9dfb291b30'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.960306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382849-41e1-4eac-b01f-3bfcd0cacb96 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:57.966991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-30e327d1-148e-416b-a458-e9c5277f90a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdd227aff83453bac3abf9dfb291b30'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:57.968513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:57.970063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:57.970968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:57.971496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:57.972715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:57.975287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:57.990124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:57.997073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 997/1964] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:29:57 2022] DELETE /identity/v3/users/1bdd227aff83453bac3abf9dfb291b30 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:57.998707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 968/1965] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:58.000074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.004626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.005021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.005603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.017604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.035833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:58.037153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.055054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1511f-c57f-49a8-a07e-4760555c0add admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.183278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 998/1966] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:58.184462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.190924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.206519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a852e0-33ba-4ddc-aca1-8e6cd358e3ea None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.213389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2f1c3-49fb-453c-a758-14f4b67d0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.214326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f1c3-49fb-453c-a758-14f4b67d0441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.214751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f1c3-49fb-453c-a758-14f4b67d0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.215247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f1c3-49fb-453c-a758-14f4b67d0441 admin admin] PATH_INFO: `/v3/projects/8a7583ba93364d578a78c9223b99b192/users/ab0990c938bc4449b2aafcd1c053aab6/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.246946 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f1c3-49fb-453c-a758-14f4b67d0441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7583ba93364d578a78c9223b99b192, user_id=ab0990c938bc4449b2aafcd1c053aab6, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.251993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f1c3-49fb-453c-a758-14f4b67d0441 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.270107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.273896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.278640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.279078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] PATH_INFO: `/v3/projects/96365a939354418f94b8c6517bdd2c3b/users/2d5497cbdefa4a8dabc5649a55642b67/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.281166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 999/1967] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:58 2022] PUT /identity/v3/projects/8a7583ba93364d578a78c9223b99b192/users/ab0990c938bc4449b2aafcd1c053aab6/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:58.283502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f8ca-ff04-4d40-823b-071ece13e76c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.283791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f8ca-ff04-4d40-823b-071ece13e76c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.284160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f8ca-ff04-4d40-823b-071ece13e76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.311293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a508f8ca-ff04-4d40-823b-071ece13e76c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:58.313228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a508f8ca-ff04-4d40-823b-071ece13e76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.336078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96365a939354418f94b8c6517bdd2c3b, user_id=2d5497cbdefa4a8dabc5649a55642b67, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.344429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.367946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 969/1968] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:57 2022] PUT /identity/v3/projects/96365a939354418f94b8c6517bdd2c3b/users/2d5497cbdefa4a8dabc5649a55642b67/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:58.369698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.380139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:58.423032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1000/1969] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:58.424412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.424700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.425216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.487518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] MFA Rules not processed for user `2d5497cbdefa4a8dabc5649a55642b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:58.488897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.492534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bac94-cd7a-46ce-b339-dccfd4cc2715 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.517095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.518530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.518748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.519528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.526117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.532820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.547473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 970/1970] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:58.550146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.566020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c415667-7fc2-4a90-97cb-e3f21f74eb01 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.572904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.573679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.574482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.574482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.575399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:58.575808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.583753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.603569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 971/1971] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:58.605139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.607312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.618303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1001/1972] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:58 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:58.620880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.627150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3f42b-5321-48e1-a3e9-c9e35b463208 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.632638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.633609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.633880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.634156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] PATH_INFO: `/v3/projects/66677dc61a194ea68885ddac59bb2579` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.638205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66677dc61a194ea68885ddac59bb2579)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.641441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.645027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359cb62-a405-4200-b9a6-b802e1a97dc4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.656337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.657597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.658179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.658758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] PATH_INFO: `/v3/projects/8a7583ba93364d578a78c9223b99b192/users/ab0990c938bc4449b2aafcd1c053aab6/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.664264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7583ba93364d578a78c9223b99b192, user_id=ab0990c938bc4449b2aafcd1c053aab6, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.670689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.682066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:58.717244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1002/1973] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:58 2022] PUT /identity/v3/projects/8a7583ba93364d578a78c9223b99b192/users/ab0990c938bc4449b2aafcd1c053aab6/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:58.718934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.757741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66677dc61a194ea68885ddac59bb2579', 'request_id': 'req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:58.769494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66677dc61a194ea68885ddac59bb2579', 'request_id': 'req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:58.785926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66677dc61a194ea68885ddac59bb2579', 'request_id': 'req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:58.810348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] The token cache is being invalidate because project 66677dc61a194ea68885ddac59bb2579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:29:58.810694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:58.811608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:29:58.814068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 972/1974] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:29:58 2022] DELETE /identity/v3/projects/66677dc61a194ea68885ddac59bb2579 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:58.814942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:58.816647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:58.890487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dd220-f7a2-4107-bb82-68e6e8e89508 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.956436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9e6d9-872a-4afe-a06d-6d0e1ce07933 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:58.969779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e495c50-39c6-4a06-b4fe-d5d3f5304ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:58.970844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e495c50-39c6-4a06-b4fe-d5d3f5304ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:58.971507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e495c50-39c6-4a06-b4fe-d5d3f5304ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:58.972058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e495c50-39c6-4a06-b4fe-d5d3f5304ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:58.973600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e495c50-39c6-4a06-b4fe-d5d3f5304ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:58.979671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e495c50-39c6-4a06-b4fe-d5d3f5304ba0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:58.998906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28924121-c4dc-4bf6-924b-2d4968488bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.000436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28924121-c4dc-4bf6-924b-2d4968488bea admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.003430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28924121-c4dc-4bf6-924b-2d4968488bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.003759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28924121-c4dc-4bf6-924b-2d4968488bea admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.004471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28924121-c4dc-4bf6-924b-2d4968488bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.007390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28924121-c4dc-4bf6-924b-2d4968488bea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.028108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:59.050338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 973/1975] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:58 2022] POST /identity/v3/projects => generated 407 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:59.050338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.050338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.050338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.095718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1003/1976] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:58 2022] POST /identity/v3/projects => generated 379 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:59.100351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.101265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.101546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.124520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] MFA Rules not processed for user `ab0990c938bc4449b2aafcd1c053aab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:59.125992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:59.151836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:59.154812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:59.236966 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 974/1977] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:59.238074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.345052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1004/1978] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:59.346491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.348132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:59.364395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc59d5-b8f5-4f4d-a09a-c66f5d1ccc50 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.371755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.372331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.372714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.372985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.373866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.376141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.395717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214998b-55fa-41e5-862b-229d340c3a70 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.403823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b577836-4249-485e-995f-0ca50d6b22fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.404669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b577836-4249-485e-995f-0ca50d6b22fd admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.405227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b577836-4249-485e-995f-0ca50d6b22fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.405759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b577836-4249-485e-995f-0ca50d6b22fd admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.409072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b577836-4249-485e-995f-0ca50d6b22fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.409269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1005/1979] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/users => generated 290 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:59.409269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.409441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b577836-4249-485e-995f-0ca50d6b22fd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.413635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:59.435404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:59.448792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 975/1980] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/users => generated 310 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:29:59.451349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.452039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.452541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.490265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:29:59.492553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:59.505109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 976/1981] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:29:59.509295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.511635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc5d0-fd06-4adf-96dc-1b6962153d79 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.516929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83853eaa-c538-4f1c-9b07-17a7d4260c94 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.519873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.521117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.521587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.522063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.522186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.522808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.523154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.523489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.524220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.526202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.538596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.540081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.544608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 977/1982] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:59.546396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.561315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1006/1983] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:59 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:59.563234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.565530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec77442-5232-4580-860e-f67420562c59 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.574877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.575989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.576344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.576719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.581038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.581038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.597208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 978/1984] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:59.597208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.597542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10ad4b-fdfa-4b07-86d6-de41dee1acb5 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.608954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.610474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.611157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.611763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.613147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:29:59.613414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc1a7e-082c-474f-b6dd-ee9fa4e0e542 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.613961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.617387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.622491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.623171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.623458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.623703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.624746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.627614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.645167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 979/1985] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:29:59.645167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.648268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1007/1986] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:29:59.649426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.663468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81897a-f361-458c-a8a7-c799569b14c6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.669477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.670422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.671010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.671356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] PATH_INFO: `/v3/projects/b0dcf43b6aae4d9e9855fa35568421f8/users/96de3918cda64c80939bc627b87e0b49/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.699269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd870c2b-b53d-42ca-8654-bc1838690827 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.710018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.711071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.711531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.711932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] PATH_INFO: `/v3/projects/4f6a5a52e6ff42979fe588c3b878408e/users/bb4b7d3da4a24c0ba7408dd97ec54800/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.715877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0dcf43b6aae4d9e9855fa35568421f8, user_id=96de3918cda64c80939bc627b87e0b49, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.722142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.735388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6a5a52e6ff42979fe588c3b878408e, user_id=bb4b7d3da4a24c0ba7408dd97ec54800, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.740467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.761181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1008/1987] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:59 2022] PUT /identity/v3/projects/b0dcf43b6aae4d9e9855fa35568421f8/users/96de3918cda64c80939bc627b87e0b49/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:59.761181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.772431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 980/1988] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:29:59 2022] PUT /identity/v3/projects/4f6a5a52e6ff42979fe588c3b878408e/users/bb4b7d3da4a24c0ba7408dd97ec54800/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:29:59.776910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:29:59.798248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:29:59.876182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41e157-43ee-409d-9b84-1305118b709a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.881868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:29:59.882501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.882777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.883047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:29:59.885951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:29:59.898631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680dd4c8-6dfc-49a1-8851-4eaa21d78886 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:29:59.980514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:29:59.982276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad09d6-5287-4b4a-bd11-34b72d5c76f8 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:29:59.993560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 981/1989] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:29:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:29:59.994818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:29:59.995258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:29:59.995460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.037458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] MFA Rules not processed for user `96de3918cda64c80939bc627b87e0b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:00.039404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:00.065751 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.066548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.066832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.067111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.068484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.071347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.128750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1009/1990] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/projects => generated 377 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:00.128750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.140235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:00.184162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 982/1991] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:00.191081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.229245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef3382-caab-4e39-bf91-3793d427bc80 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.239461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e64b1-a5ad-4a5d-aff4-b589deef988d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.247873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.248748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.249217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.249603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.250764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.254945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.256421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.258507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.258836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.259703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.264466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.266423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.313480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1010/1992] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:00.314550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.357629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341ab63-9234-4f25-a3c0-2937295a7eca admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.363397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 983/1993] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:00 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:00.363397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.372140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.372784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.375689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.375983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.378803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.383363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.446287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b42f1-768c-4b2b-a5c8-8a280c9d43d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.476788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1011/1994] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:00 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:00.477226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.478133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.478509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.478930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] PATH_INFO: `/v3/projects/4f6a5a52e6ff42979fe588c3b878408e/users/bb4b7d3da4a24c0ba7408dd97ec54800/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.482609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6a5a52e6ff42979fe588c3b878408e, user_id=bb4b7d3da4a24c0ba7408dd97ec54800, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.489383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.495715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.537550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 984/1995] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:00 2022] PUT /identity/v3/projects/4f6a5a52e6ff42979fe588c3b878408e/users/bb4b7d3da4a24c0ba7408dd97ec54800/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:00.549313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.615702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:00.680866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051a8a5-2a0f-432e-b9e4-841e1288e569 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.694524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e00cef-f0fd-425f-980a-33a5ed233ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.695175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e00cef-f0fd-425f-980a-33a5ed233ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.695466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e00cef-f0fd-425f-980a-33a5ed233ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.695719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e00cef-f0fd-425f-980a-33a5ed233ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.697082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e00cef-f0fd-425f-980a-33a5ed233ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.698643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e00cef-f0fd-425f-980a-33a5ed233ca1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.708718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217702b3-e628-4ef5-aac5-3fb049b70168 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.722728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.724113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.725275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.726341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.727367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.730503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.733464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1012/1996] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:00.736709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963cc1c-3438-4ae2-99a5-286d2124cab6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.737977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963cc1c-3438-4ae2-99a5-286d2124cab6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.738322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963cc1c-3438-4ae2-99a5-286d2124cab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.773386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 985/1997] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:00 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:00.775888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.809079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83f750-74fb-4528-ac74-7182bb9e8f40 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.819527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.822189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.822590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.823007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.823793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.825920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.865921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 986/1998] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:00.868230 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.886155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9ed5-c420-4ff5-8675-c002d920d16f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.893534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.894353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.894776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.895165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.896042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.897864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:00.905644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0963cc1c-3438-4ae2-99a5-286d2124cab6 None None] MFA Rules not processed for user `bb4b7d3da4a24c0ba7408dd97ec54800`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:00.908072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0963cc1c-3438-4ae2-99a5-286d2124cab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:00.916766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 987/1999] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:00.917745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:00.929431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c5db9-e7f4-4133-9095-06ba59e5d600 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:00.935584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:00.936580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:00.937740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:00.937740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] PATH_INFO: `/v3/projects/5a67532f113c4416a39830450683fb61/users/48ff09b1c6bc4d1588e4300ee547f8ba/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:00.965328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a67532f113c4416a39830450683fb61, user_id=48ff09b1c6bc4d1588e4300ee547f8ba, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:00.969084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:01.016978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 988/2000] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:00 2022] PUT /identity/v3/projects/5a67532f113c4416a39830450683fb61/users/48ff09b1c6bc4d1588e4300ee547f8ba/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:01.018204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:01.188376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52311e-3d04-4418-89b1-1da54aaec02f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:01.201851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:01.203857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:01.204558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:01.204981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] PATH_INFO: `/v3/projects/92a51a1234c3450faaab80c644acadf5/users/9b2b83f7c73e480499a17f900d3ab9f8/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:01.209463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1013/2001] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:01.209463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:01.209463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:01.210112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:01.240901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:01.257111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a51a1234c3450faaab80c644acadf5, user_id=9b2b83f7c73e480499a17f900d3ab9f8, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:01.265629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:01.291112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] MFA Rules not processed for user `48ff09b1c6bc4d1588e4300ee547f8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:01.292444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:01.297036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 989/2002] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:01 2022] PUT /identity/v3/projects/92a51a1234c3450faaab80c644acadf5/users/9b2b83f7c73e480499a17f900d3ab9f8/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:01.303860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:01.307868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:01.400981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1014/2003] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:01.402227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:01.490998 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:01.503902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd017457-cbfa-4b13-be5d-011eb35e328b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:01.522508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:01.523741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:01.525439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:01.527104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:01.545692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f8906-9bd4-41f3-9bef-0279e2c9a56d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:01.560817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:01.561553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:01.562108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:01.562819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:01.565829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:01.569164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:01.598334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1015/2004] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:01.599971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:01.606565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:01.652391 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:01.654256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:01.671768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 990/2005] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:01.687016 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:01.718055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa32219-1654-4099-aa2f-30770ce54414 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:01.742163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e6e77-7543-4bcf-9cf1-807f2644c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:01.746928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e6e77-7543-4bcf-9cf1-807f2644c396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:01.757277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e6e77-7543-4bcf-9cf1-807f2644c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:01.757860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e6e77-7543-4bcf-9cf1-807f2644c396 admin admin] PATH_INFO: `/v3/projects/92a51a1234c3450faaab80c644acadf5/users/9b2b83f7c73e480499a17f900d3ab9f8/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:01.763046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e6e77-7543-4bcf-9cf1-807f2644c396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a51a1234c3450faaab80c644acadf5, user_id=9b2b83f7c73e480499a17f900d3ab9f8, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:01.772988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e6e77-7543-4bcf-9cf1-807f2644c396 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:01.840476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 991/2006] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:01 2022] PUT /identity/v3/projects/92a51a1234c3450faaab80c644acadf5/users/9b2b83f7c73e480499a17f900d3ab9f8/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:01.853363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:01.853860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:01.854330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:01.882448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff390b-70da-4a16-b4b5-d6470a89df49 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:01.936877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:01.949885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] MFA Rules not processed for user `9b2b83f7c73e480499a17f900d3ab9f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:01.952734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:02.008194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:02.012230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:02.013622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:02.014120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:02.163814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:02.169149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:02.185887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1016/2007] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:02.209070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 992/2008] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:02.221578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:02.307174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7b5fe-4570-44ca-b7ec-5234fb92c2bc service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:02.312063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:02.312787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:02.313237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:02.313456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:02.315610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:02.436944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:02.440601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:02.462266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1017/2009] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:30:03.026088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:03.072332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9f0e2-32e7-4cf6-886c-f719a62f9275 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:03.086793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:03.090129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:03.090129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:03.090470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:03.136426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:03.144267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:03.169957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 993/2010] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:04.598535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:04.611370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216df6-9fcc-4ae8-98bf-43537819cea7 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:04.615826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:04.616420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:04.616985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:04.617765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:04.620637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:04.675382 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:04.738387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:04.739961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41889-5171-44ad-96b8-af2eca4fda9b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:04.766116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1018/2011] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:05.644456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:05.727010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:05.736354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538a320-d544-467f-8dfa-22e6d6225572 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:05.744823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:05.745884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:05.746335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:05.746749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] PATH_INFO: `/v3/users/96de3918cda64c80939bc627b87e0b49` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:05.748190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=96de3918cda64c80939bc627b87e0b49)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:05.750650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:05.813487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96de3918cda64c80939bc627b87e0b49'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:05.823020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96de3918cda64c80939bc627b87e0b49'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:05.823721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96de3918cda64c80939bc627b87e0b49'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:05.824442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96de3918cda64c80939bc627b87e0b49'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:05.833935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96de3918cda64c80939bc627b87e0b49'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:05.872219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 994/2012] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:05 2022] DELETE /identity/v3/users/96de3918cda64c80939bc627b87e0b49 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:05.878795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:05.879924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:05.879924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:05.906262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:05.908028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:06.008446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1019/2013] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:06.024105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:06.026685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:06.150837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1707-b249-4d27-a122-b1cfbfd3f61e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:06.157828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:06.158912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:06.159414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:06.159909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:06.176823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:06.179557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:06.203620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 995/2014] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:30:06.625345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:06.627518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:06.645537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d2af-8506-48b6-87ef-20760a335ef6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:06.653293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:06.654815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:06.656016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:06.656822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] PATH_INFO: `/v3/projects/b0dcf43b6aae4d9e9855fa35568421f8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:06.659791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0dcf43b6aae4d9e9855fa35568421f8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:06.664528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:06.748300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0dcf43b6aae4d9e9855fa35568421f8', 'request_id': 'req-fea7f483-0159-4f1a-a7fd-74d7d56f123a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.756589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0dcf43b6aae4d9e9855fa35568421f8', 'request_id': 'req-fea7f483-0159-4f1a-a7fd-74d7d56f123a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.766916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0dcf43b6aae4d9e9855fa35568421f8', 'request_id': 'req-fea7f483-0159-4f1a-a7fd-74d7d56f123a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.786118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] The token cache is being invalidate because project b0dcf43b6aae4d9e9855fa35568421f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:06.786502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.787625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.790340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1020/2015] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:06 2022] DELETE /identity/v3/projects/b0dcf43b6aae4d9e9855fa35568421f8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:06.796749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:06.814389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f4b9b-57bc-43e4-9ba8-ac62f8cc395b service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:06.896203 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:06.896991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:06.897311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:06.897586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] PATH_INFO: `/v3/users/9b2b83f7c73e480499a17f900d3ab9f8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:06.899154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2b83f7c73e480499a17f900d3ab9f8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:06.902164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:06.928575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:06.964017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2b83f7c73e480499a17f900d3ab9f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.974271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2b83f7c73e480499a17f900d3ab9f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.975285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2b83f7c73e480499a17f900d3ab9f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.976217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2b83f7c73e480499a17f900d3ab9f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:06.984926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-859e5f35-7ee4-4922-91ef-4f28afdcff82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2b83f7c73e480499a17f900d3ab9f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:07.006789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 996/2016] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:06 2022] DELETE /identity/v3/users/9b2b83f7c73e480499a17f900d3ab9f8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:07.376361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:07.378637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:07.403237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f483-0159-4f1a-a7fd-74d7d56f123a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:07.508603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:07.520072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:07.521096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:07.521645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:07.522047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] PATH_INFO: `/v3/projects/92a51a1234c3450faaab80c644acadf5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:07.523662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a51a1234c3450faaab80c644acadf5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:07.526020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:07.614262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a51a1234c3450faaab80c644acadf5', 'request_id': 'req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:07.625548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a51a1234c3450faaab80c644acadf5', 'request_id': 'req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:07.638944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a51a1234c3450faaab80c644acadf5', 'request_id': 'req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:07.660161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] The token cache is being invalidate because project 92a51a1234c3450faaab80c644acadf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:07.660728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:07.661788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce7e0247-9ca7-4a1b-90c7-dafb8de25556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:07.664486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1021/2017] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:07 2022] DELETE /identity/v3/projects/92a51a1234c3450faaab80c644acadf5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:07.670936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:07.671416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:07.674178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:07.704942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:07.706651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:07.798298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 997/2018] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:07.813237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:07.813237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:07.813237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:07.852054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:07.854290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:07.864646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1022/2019] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:07.869852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:07.882223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469f603-b9b0-4efd-9e7e-2abab9f841ee None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:07.890084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:07.891067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:07.891534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:07.891995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:07.893139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:07.893660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:07.896632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:07.926659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 998/2020] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:30:07.927742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:07.945749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613a6a-a66c-4ae7-b246-eac2892e2c16 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:07.951031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:07.951848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:07.952263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:07.952670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:07.953787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:07.956444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:07.990431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1023/2021] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:07 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:07.995743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:08.010218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17dfe7-391a-4a1d-b540-ef38c39e5374 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:08.084217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:08.085100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.085390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.085716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.086549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:08.089236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:08.115054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 999/2022] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:07 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:08.122004 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:08.143078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343952d8-1066-469a-807e-b7aef4457a1a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:08.152775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:08.154253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.154868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.155384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.156776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:08.161264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:08.176925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1024/2023] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:08.183726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:08.197927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32305b82-90f1-4f93-82ef-cd53ae57a465 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:08.204991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f454b-d7da-44e1-966e-e8d5f31b6287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:08.205686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f454b-d7da-44e1-966e-e8d5f31b6287 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.206178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f454b-d7da-44e1-966e-e8d5f31b6287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.206509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f454b-d7da-44e1-966e-e8d5f31b6287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.207369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f454b-d7da-44e1-966e-e8d5f31b6287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:08.209071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f454b-d7da-44e1-966e-e8d5f31b6287 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:08.225936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1000/2024] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:08.230389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:08.251035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90418f33-317c-4279-b90f-536788b1556f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:08.261350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:08.262534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.263060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.263515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] PATH_INFO: `/v3/projects/f3f7c0735b914470a27f6ed52013fda8/users/50c26766c6eb45d493eac682321c12ba/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.289431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f7c0735b914470a27f6ed52013fda8, user_id=50c26766c6eb45d493eac682321c12ba, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:08.292684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:08.310065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1025/2025] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:08 2022] PUT /identity/v3/projects/f3f7c0735b914470a27f6ed52013fda8/users/50c26766c6eb45d493eac682321c12ba/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:08.323950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.324825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.325425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.337798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:08.369594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] MFA Rules not processed for user `50c26766c6eb45d493eac682321c12ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:08.371049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:08.453857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1001/2026] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:08.470256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:08.472038 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:08.537811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b24d8-1eed-4917-b077-f2488cffc2cc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:08.541915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5265db2-d02c-4d52-9bd0-8e47f3135f97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:08.542540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265db2-d02c-4d52-9bd0-8e47f3135f97 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.542817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265db2-d02c-4d52-9bd0-8e47f3135f97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.543174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5265db2-d02c-4d52-9bd0-8e47f3135f97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.552706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5265db2-d02c-4d52-9bd0-8e47f3135f97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:08.554258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5265db2-d02c-4d52-9bd0-8e47f3135f97 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:08.566363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1026/2027] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:08.590131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:08.680445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdec5d-d05e-4f5e-874c-b16b46b5cc07 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:08.693863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:08.694590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.694940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.695239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] PATH_INFO: `/v3/users/50c26766c6eb45d493eac682321c12ba` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:08.697508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c26766c6eb45d493eac682321c12ba)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:08.699999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:08.812612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c26766c6eb45d493eac682321c12ba'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:08.826777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c26766c6eb45d493eac682321c12ba'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:08.828191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c26766c6eb45d493eac682321c12ba'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:08.828943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c26766c6eb45d493eac682321c12ba'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:08.854334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c26766c6eb45d493eac682321c12ba'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:08.921121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1002/2028] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:08 2022] DELETE /identity/v3/users/50c26766c6eb45d493eac682321c12ba => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:08.926781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:08.927623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:08.928353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:09.007936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:09.012553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:09.057315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:09.154581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1027/2029] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:09.170335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:09.172688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:09.236565 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:09.338557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12837b0-8809-4048-b854-4e6c25f5532d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:09.355621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b441bd-48eb-4b53-a265-c6464e8b03e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:09.356790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b441bd-48eb-4b53-a265-c6464e8b03e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:09.357107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b441bd-48eb-4b53-a265-c6464e8b03e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:09.357528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b441bd-48eb-4b53-a265-c6464e8b03e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:09.379318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:09.381522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b441bd-48eb-4b53-a265-c6464e8b03e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:09.383420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b441bd-48eb-4b53-a265-c6464e8b03e5 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:09.385706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:09.398751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1003/2030] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:09.450050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93aeae-4434-426a-ae76-1c93dc5516c9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:09.479828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:09.482474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:09.483617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:09.485129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] PATH_INFO: `/v3/users/2303f076041b471cbb79dd094c4bf476` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:09.489299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2303f076041b471cbb79dd094c4bf476)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:09.494449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:09.631587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2303f076041b471cbb79dd094c4bf476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:09.680025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2303f076041b471cbb79dd094c4bf476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:09.685560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2303f076041b471cbb79dd094c4bf476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:09.688800 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2303f076041b471cbb79dd094c4bf476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:09.723395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2303f076041b471cbb79dd094c4bf476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:09.800591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1028/2031] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:09 2022] DELETE /identity/v3/users/2303f076041b471cbb79dd094c4bf476 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:09.802429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:09.802840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:09.803327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:09.838339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:09.840661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:09.949741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:09.952454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:09.996665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1004/2032] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:09.998978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.002943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.018680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163c85c-d7c4-4901-8fe8-f4dd495905bb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.025285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.026058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.026543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.026817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] PATH_INFO: `/v3/users/bc062f2c0d8e42748ee73cd41707745e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.028163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc062f2c0d8e42748ee73cd41707745e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.030408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.082366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911537d-d2cb-4fe3-a252-2467fac422e6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.089346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.090123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.090521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.090682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] PATH_INFO: `/v3/projects/f3f7c0735b914470a27f6ed52013fda8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.101812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f7c0735b914470a27f6ed52013fda8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.104733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.108731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc062f2c0d8e42748ee73cd41707745e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.118810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc062f2c0d8e42748ee73cd41707745e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.119490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc062f2c0d8e42748ee73cd41707745e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.120124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc062f2c0d8e42748ee73cd41707745e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.128937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc062f2c0d8e42748ee73cd41707745e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.154001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1005/2033] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:09 2022] DELETE /identity/v3/users/bc062f2c0d8e42748ee73cd41707745e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:10.154001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.158528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.176377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:10.191668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f7c0735b914470a27f6ed52013fda8', 'request_id': 'req-b96256a1-bea5-49d1-9dbe-74a52f548810'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.204916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f7c0735b914470a27f6ed52013fda8', 'request_id': 'req-b96256a1-bea5-49d1-9dbe-74a52f548810'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.221398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f7c0735b914470a27f6ed52013fda8', 'request_id': 'req-b96256a1-bea5-49d1-9dbe-74a52f548810'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.235878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:10.251385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] The token cache is being invalidate because project f3f7c0735b914470a27f6ed52013fda8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:10.251852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.253058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b96256a1-bea5-49d1-9dbe-74a52f548810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:10.261115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1029/2034] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:09 2022] DELETE /identity/v3/projects/f3f7c0735b914470a27f6ed52013fda8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:10.262798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c9c29-ef22-4e19-8517-f06a62e9de5d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.263093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c9c29-ef22-4e19-8517-f06a62e9de5d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.263374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c9c29-ef22-4e19-8517-f06a62e9de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.289309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b70c9c29-ef22-4e19-8517-f06a62e9de5d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:10.291520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b70c9c29-ef22-4e19-8517-f06a62e9de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.293816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45225858-18c8-4094-bd46-c036328d5377 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.379376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.380027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.380573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.380858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.382748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.384521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1030/2035] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:10.385759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28888ac-a4db-4435-87ee-104923571a3f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.386028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28888ac-a4db-4435-87ee-104923571a3f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.386314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28888ac-a4db-4435-87ee-104923571a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.407186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.408793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.412983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c28888ac-a4db-4435-87ee-104923571a3f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:10.414266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c28888ac-a4db-4435-87ee-104923571a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.423950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1006/2036] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:10.425290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.426940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.433187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1031/2037] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:10.439112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ba439-9ac2-4cb4-84cc-f46fc959fcf6 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.444405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.445226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.446595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.446940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.447381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.447868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.448335 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.465575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.468891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.489929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:10.491978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.502654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1032/2039] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:10.505482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1007/2039] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:10.507541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.533854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35aac3-f0d0-47e9-93a8-76764f804a2f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.539327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.540744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.541223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.541591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.542278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:10.542574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.545320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.568728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1033/2040] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:10.576517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.610735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3983af9-1da6-4145-a463-e596ab4a6eac service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.622452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.623652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.624146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.625095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.625881 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.632519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.685874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1008/2041] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:10 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:10.690164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.702020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ede6f7-07ce-4777-857b-e22d813a1214 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.818730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.820820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.821920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.822416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.824340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.828273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.872538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1034/2042] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:10 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:10.877362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.890502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f78c-5b35-447c-9c51-27d2b7bf58d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.896214 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.897113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.897222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.897547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.898245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.899946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.907141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.909094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:10.921068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1009/2043] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:10.926746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:10.930245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14c7d3-f1ca-4d48-beb5-464b6e1b880d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.938047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.938790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.939133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.939612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] PATH_INFO: `/v3/projects/36c2cfc1060040ffa358a31a42e83af4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.940897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c2cfc1060040ffa358a31a42e83af4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.943093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.943598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89dfe0-a3cf-4960-8f59-e1e0119a7207 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:10.951176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:10.952182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:10.952702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:10.953257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:10.954996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:10.958725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:10.998031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1010/2044] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:10.999010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:11.002824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:11.016583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:11.023632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1482-2fe4-4002-a383-44803ba653ca admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:11.060540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c2cfc1060040ffa358a31a42e83af4', 'request_id': 'req-1fec60a2-39b4-483d-9206-3a25fefca6de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.072317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c2cfc1060040ffa358a31a42e83af4', 'request_id': 'req-1fec60a2-39b4-483d-9206-3a25fefca6de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.088896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c2cfc1060040ffa358a31a42e83af4', 'request_id': 'req-1fec60a2-39b4-483d-9206-3a25fefca6de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.106149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] The token cache is being invalidate because project 36c2cfc1060040ffa358a31a42e83af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:11.106514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.107197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.107537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:11.108592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:11.113169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1035/2045] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:10 2022] DELETE /identity/v3/projects/36c2cfc1060040ffa358a31a42e83af4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:11.113477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:11.113573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:11.113706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] PATH_INFO: `/v3/projects/fc4c7a06b8504311a5664364f41a31b7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:11.113797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:11.114488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4c7a06b8504311a5664364f41a31b7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:11.117927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:11.125151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec60a2-39b4-483d-9206-3a25fefca6de admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:11.132065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:11.132770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:11.133083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:11.133357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] PATH_INFO: `/v3/projects/76325bc9b05344a68aa2e9a8f37bd9ee/users/61f2298168ac47b88c01b11ec1433d75/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:11.160854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76325bc9b05344a68aa2e9a8f37bd9ee, user_id=61f2298168ac47b88c01b11ec1433d75, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:11.164706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:11.178577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:11.184628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1036/2046] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:11 2022] PUT /identity/v3/projects/76325bc9b05344a68aa2e9a8f37bd9ee/users/61f2298168ac47b88c01b11ec1433d75/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:11.186346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:11.190479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:11.203144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4c7a06b8504311a5664364f41a31b7', 'request_id': 'req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.213823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4c7a06b8504311a5664364f41a31b7', 'request_id': 'req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.229091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4c7a06b8504311a5664364f41a31b7', 'request_id': 'req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.247431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] The token cache is being invalidate because project fc4c7a06b8504311a5664364f41a31b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:11.247887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.248890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2aefdb9d-b3ef-4f20-9b65-7956b9c5e3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.251747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1011/2047] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:10 2022] DELETE /identity/v3/projects/fc4c7a06b8504311a5664364f41a31b7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:11.253269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:11.253554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:11.253829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:11.295223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] MFA Rules not processed for user `61f2298168ac47b88c01b11ec1433d75`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:11.301043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:11.337474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953838e2-3398-486e-9ad0-c9d3ee37ea55 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:11.413058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1012/2048] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:11.415555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:11.418672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:11.436598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:11.437459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:11.438292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:11.438573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] PATH_INFO: `/v3/users/ab0990c938bc4449b2aafcd1c053aab6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:11.440453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0990c938bc4449b2aafcd1c053aab6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:11.443318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:11.510310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:11.530323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0990c938bc4449b2aafcd1c053aab6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.543978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e61e8-6fe1-47ea-bdcc-cfe98367a72e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:11.548509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0990c938bc4449b2aafcd1c053aab6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.549315 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0990c938bc4449b2aafcd1c053aab6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.550687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0990c938bc4449b2aafcd1c053aab6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.552520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:11.553510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:11.553844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:11.554222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] PATH_INFO: `/v3/users/91de82877474402abbc10dc220c4fbb1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:11.556431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=91de82877474402abbc10dc220c4fbb1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:11.558728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:11.584851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0990c938bc4449b2aafcd1c053aab6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.633187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1037/2049] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:11 2022] DELETE /identity/v3/users/ab0990c938bc4449b2aafcd1c053aab6 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:11.634286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:11.640077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:11.690091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91de82877474402abbc10dc220c4fbb1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.707287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:11.714530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91de82877474402abbc10dc220c4fbb1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.715914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91de82877474402abbc10dc220c4fbb1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.717930 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91de82877474402abbc10dc220c4fbb1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.735260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91de82877474402abbc10dc220c4fbb1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:11.762146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1013/2050] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:11 2022] DELETE /identity/v3/users/91de82877474402abbc10dc220c4fbb1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:11.797884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b380e94-6b02-4a31-8599-bbdea4535dd2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:11.932948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:11.934013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:11.934381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:11.934688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:11.940318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:12.098084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:12.103331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:12.117672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1038/2051] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:12.259799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:12.259799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:12.342742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:12.344200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:12.421268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:12.424910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b9e83-e663-4cdf-ae38-27be8af85c2d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:12.437724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:12.438382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.438657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.438950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] PATH_INFO: `/v3/projects/8a7583ba93364d578a78c9223b99b192` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.442990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7583ba93364d578a78c9223b99b192)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:12.448483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:12.488181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246b4b9-a850-419a-b5f6-a2f56214d902 service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:12.499916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:12.499916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.499916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.499916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] PATH_INFO: `/v3/projects/77bf920f924f424795abd93b421afa69` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.501944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bf920f924f424795abd93b421afa69)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:12.505746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:12.550735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7583ba93364d578a78c9223b99b192', 'request_id': 'req-bbf39622-3074-4129-a92d-e61400c62a51'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.559866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7583ba93364d578a78c9223b99b192', 'request_id': 'req-bbf39622-3074-4129-a92d-e61400c62a51'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.572540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7583ba93364d578a78c9223b99b192', 'request_id': 'req-bbf39622-3074-4129-a92d-e61400c62a51'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.573983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bf920f924f424795abd93b421afa69', 'request_id': 'req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.587422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bf920f924f424795abd93b421afa69', 'request_id': 'req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.596979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] The token cache is being invalidate because project 8a7583ba93364d578a78c9223b99b192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:12.598189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.599272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bbf39622-3074-4129-a92d-e61400c62a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.603904 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bf920f924f424795abd93b421afa69', 'request_id': 'req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.609980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1014/2052] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:12 2022] DELETE /identity/v3/projects/8a7583ba93364d578a78c9223b99b192 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:12.611494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.611773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.612042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.625627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] The token cache is being invalidate because project 77bf920f924f424795abd93b421afa69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:12.626391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.627862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.631361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1039/2053] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:12 2022] DELETE /identity/v3/projects/77bf920f924f424795abd93b421afa69 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:12.636840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:12.637548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:12.640122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:12.640293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:12.740728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:12.766303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1015/2054] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:12.766303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:12.766738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a26ccc-e365-4d08-bf9d-3489f6d1fc61 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:12.766919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:12.767289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:12.768162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.768464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.768775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] PATH_INFO: `/v3/users/9edca9ea031748a1bbae78de7b4336fb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.770235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9edca9ea031748a1bbae78de7b4336fb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:12.772195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:12.780049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16c8e-efbb-442d-ad81-c1d6684421b9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:12.796853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:12.798106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.798746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.799269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] PATH_INFO: `/v3/users/2d5497cbdefa4a8dabc5649a55642b67` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.801168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5497cbdefa4a8dabc5649a55642b67)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:12.803603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:12.872563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9edca9ea031748a1bbae78de7b4336fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.895170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5497cbdefa4a8dabc5649a55642b67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.902700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9edca9ea031748a1bbae78de7b4336fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.903749 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9edca9ea031748a1bbae78de7b4336fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.904609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9edca9ea031748a1bbae78de7b4336fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.912876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5497cbdefa4a8dabc5649a55642b67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.913737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5497cbdefa4a8dabc5649a55642b67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.914363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5497cbdefa4a8dabc5649a55642b67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.920219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:12.925313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2dd0c6ac-adcb-4c8c-ab7d-d2a882ea7d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9edca9ea031748a1bbae78de7b4336fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.925548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-35478a7e-9042-4874-8772-1c34bf382314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5497cbdefa4a8dabc5649a55642b67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:12.956103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1016/2055] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:12 2022] DELETE /identity/v3/users/2d5497cbdefa4a8dabc5649a55642b67 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:12.957669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.957978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.958240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.966564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1040/2056] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:12 2022] DELETE /identity/v3/users/9edca9ea031748a1bbae78de7b4336fb => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:12.969202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:12.969202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:12.969202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:12.991658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:12.994360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.001058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:13.003112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.203357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1041/2057] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:13.221299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.224682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.275129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1017/2058] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:13.278042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.308715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ba1f8-b771-4f0b-8aeb-5790f3cb6439 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.323590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.327497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.328063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.328404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.329571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:13.330026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.331664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.353213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1018/2059] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:30:13.354635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512c547-ef67-43f3-a697-8c731a6d590d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.357803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.358667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.359338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.359738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.359887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.370482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfead-2de1-46c6-823f-17a9fd5a0205 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.373688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.375774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.377842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.378566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.378914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.379336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.379979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.382993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.396954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1042/2060] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:13.422581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1019/2061] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:13 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:13.426776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.443159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5182-49c1-4466-84bd-808eebf630e6 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.498155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.504972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.556448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.556448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.556448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.557046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.557917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.560549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.568787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:13.602974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1043/2062] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:13 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:13.608392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.616844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921480f4-594f-4906-bee8-44039be5c3de admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.624955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.625515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f029a-cecb-411e-81f6-81b92af73326 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.626079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.626568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.626950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] PATH_INFO: `/v3/projects/f1161df8fd6b482b8f937107721ba4e1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.628496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1161df8fd6b482b8f937107721ba4e1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.630868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.646884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.647570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.647914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.648106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.648763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.650660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.653027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:13.670498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1044/2063] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:13.671585 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.685802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fcc09-8c7c-40ed-882f-0855e91e68f7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.701936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1161df8fd6b482b8f937107721ba4e1', 'request_id': 'req-834ed69a-f49a-48e4-8d33-79c038555735'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:13.710641 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1161df8fd6b482b8f937107721ba4e1', 'request_id': 'req-834ed69a-f49a-48e4-8d33-79c038555735'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:13.725453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1161df8fd6b482b8f937107721ba4e1', 'request_id': 'req-834ed69a-f49a-48e4-8d33-79c038555735'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:13.751781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] The token cache is being invalidate because project f1161df8fd6b482b8f937107721ba4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:13.752137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:13.752893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-834ed69a-f49a-48e4-8d33-79c038555735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:13.757046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1020/2064] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:13 2022] DELETE /identity/v3/projects/f1161df8fd6b482b8f937107721ba4e1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:13.767274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.767555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.767835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.815473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b108b2-80c1-4920-a5ec-7012afe5302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.816683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b108b2-80c1-4920-a5ec-7012afe5302f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.818301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b108b2-80c1-4920-a5ec-7012afe5302f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.818301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b108b2-80c1-4920-a5ec-7012afe5302f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.818777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:13.819841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b108b2-80c1-4920-a5ec-7012afe5302f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.820481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.824150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b108b2-80c1-4920-a5ec-7012afe5302f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.850096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1021/2065] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:13.851188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.853596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.859670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1045/2066] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:13.867361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.867681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.867980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.874051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58467ae5-efbc-4370-b0f5-5a82c75fc312 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:13.882475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:13.883506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:13.883903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:13.884432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] PATH_INFO: `/v3/projects/96365a939354418f94b8c6517bdd2c3b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:13.888669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96365a939354418f94b8c6517bdd2c3b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:13.891208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:13.903275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:13.908509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.938739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1046/2067] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:13.939446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:13.940790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:13.962364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3301d3-ab5c-4ba6-bed1-8d0d7228f764 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.002675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96365a939354418f94b8c6517bdd2c3b', 'request_id': 'req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.012694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96365a939354418f94b8c6517bdd2c3b', 'request_id': 'req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.025878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96365a939354418f94b8c6517bdd2c3b', 'request_id': 'req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.041371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] The token cache is being invalidate because project 96365a939354418f94b8c6517bdd2c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:14.041664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.042436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.044250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1022/2068] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:13 2022] DELETE /identity/v3/projects/96365a939354418f94b8c6517bdd2c3b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:14.047999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:14.049582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:14.058288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.059466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.060143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.060544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] PATH_INFO: `/v3/projects/293a09cd5b4e4d8b9cfa67ba52c9366c/users/98f0b63b9c1e4a459bdf8d30e2efc23c/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.078986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293a09cd5b4e4d8b9cfa67ba52c9366c, user_id=98f0b63b9c1e4a459bdf8d30e2efc23c, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.082268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.107399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1047/2069] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:13 2022] PUT /identity/v3/projects/293a09cd5b4e4d8b9cfa67ba52c9366c/users/98f0b63b9c1e4a459bdf8d30e2efc23c/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:14.107849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:14.109507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:14.110126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:14.142686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed76c-dacd-4e4b-9d4d-4fb61ac8d006 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.233966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922f99-2a16-416a-8240-0648efeb3071 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.240295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.241397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.241864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.256767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] PATH_INFO: `/v3/users/bb4b7d3da4a24c0ba7408dd97ec54800` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.262000 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4b7d3da4a24c0ba7408dd97ec54800)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.263374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.268933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.270667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.271051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.271425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.272227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:14.272631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.275965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4b52-1d34-4f6e-8110-a10364fb6911 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.291709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:14.310542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1023/2070] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:14.329439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.329762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.330103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.399193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] MFA Rules not processed for user `98f0b63b9c1e4a459bdf8d30e2efc23c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:14.400487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:14.428342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4b7d3da4a24c0ba7408dd97ec54800'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.445695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4b7d3da4a24c0ba7408dd97ec54800'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.447243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4b7d3da4a24c0ba7408dd97ec54800'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.448419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4b7d3da4a24c0ba7408dd97ec54800'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.462051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4b7d3da4a24c0ba7408dd97ec54800'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:14.503795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1048/2071] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:14 2022] DELETE /identity/v3/users/bb4b7d3da4a24c0ba7408dd97ec54800 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:14.504548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:14.506300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:14.529799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1024/2072] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:14.538372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:14.541974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:14.571710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1efdb-b5b8-41b7-9e9d-8c1622086298 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.725337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068340f2-0855-4317-9617-a9c7173046b1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.731017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.732012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.732012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.732305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.733050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.735082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.764251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.770515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.770515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.770515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.770515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.776381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.806180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1049/2073] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:14 2022] POST /identity/v3/projects => generated 391 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:14.807309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:14.829395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:14.839683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1025/2074] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:14 2022] POST /identity/v3/projects => generated 385 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:14.844504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:14.949647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88317-58e0-45ac-88c5-04ff1a940f0e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.953345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c83c-0114-4099-b2b3-4df2c097d0e8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:14.957639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.958570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.959740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.960147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:14.960399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.960940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:14.961251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:14.961566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:14.962143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.962273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:14.965177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.967949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:14.998780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1026/2075] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:14 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:15.000288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.010193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1050/2076] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:14 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:15.010668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.015855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207132d3-dc5a-4d24-8674-bafab41e8435 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.026014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71c8ef-c937-41ca-9f60-e0912bb8f44f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.028320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:15.036286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.036542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.036777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.037242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.040368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.041180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.043025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.044352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.044640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.044952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.046541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.050561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.070408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1051/2077] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:15.082346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.088544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1027/2078] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:15.088544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.092120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:15.102930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7f9fd-c088-4e7c-9e7d-3dbd649cb6d4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.126008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.127293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.127658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.127996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.128919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.131185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.135301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b8723-5df7-48b7-999e-94cc58ee3e98 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.142517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.143181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.143460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.143734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] PATH_INFO: `/v3/projects/4f6a5a52e6ff42979fe588c3b878408e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.145544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6a5a52e6ff42979fe588c3b878408e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.149051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1052/2079] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:15.149051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.149303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.158742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c1c8-b38f-44eb-b282-d9003c439049 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.164272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.165869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.166445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.166445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] PATH_INFO: `/v3/projects/69bbd9a7e9db4fda8145328616b2d2f1/users/389fb0c4f26142fdbd9d2ae6499be7b1/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.186133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bbd9a7e9db4fda8145328616b2d2f1, user_id=389fb0c4f26142fdbd9d2ae6499be7b1, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.188980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.213827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1053/2080] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:15 2022] PUT /identity/v3/projects/69bbd9a7e9db4fda8145328616b2d2f1/users/389fb0c4f26142fdbd9d2ae6499be7b1/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:15.213827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.229996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6a5a52e6ff42979fe588c3b878408e', 'request_id': 'req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:15.237649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6a5a52e6ff42979fe588c3b878408e', 'request_id': 'req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:15.247811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6a5a52e6ff42979fe588c3b878408e', 'request_id': 'req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:15.263550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] The token cache is being invalidate because project 4f6a5a52e6ff42979fe588c3b878408e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:15.263797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:15.264497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:15.266281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1028/2081] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:15 2022] DELETE /identity/v3/projects/4f6a5a52e6ff42979fe588c3b878408e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:15.267260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.269584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:15.326832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbd800-4ba9-4f30-af88-8511b56e68b3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.442454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaa141-cc72-4ebb-b39c-04cb93ea6b6c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.457328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dae12-082c-4717-874d-404b7451464c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.457984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dae12-082c-4717-874d-404b7451464c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.458264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dae12-082c-4717-874d-404b7451464c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.458552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dae12-082c-4717-874d-404b7451464c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.459318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dae12-082c-4717-874d-404b7451464c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.461090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dae12-082c-4717-874d-404b7451464c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.472030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:15.484852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.485668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.485951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.486254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] PATH_INFO: `/v3/projects/9bfe702cbe64436baf89da2e56ba1014/users/5d7447e6c7db45c49ef11f7abfe4dde3/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.509659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfe702cbe64436baf89da2e56ba1014, user_id=5d7447e6c7db45c49ef11f7abfe4dde3, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.514837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.518113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1029/2082] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:15.523403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02147a1b-5323-445e-a87f-8dd18c87c1e5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.524942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02147a1b-5323-445e-a87f-8dd18c87c1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.525516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02147a1b-5323-445e-a87f-8dd18c87c1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.560290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1054/2083] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:15 2022] PUT /identity/v3/projects/9bfe702cbe64436baf89da2e56ba1014/users/5d7447e6c7db45c49ef11f7abfe4dde3/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:15.560290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:15.590554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-02147a1b-5323-445e-a87f-8dd18c87c1e5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:15.592506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-02147a1b-5323-445e-a87f-8dd18c87c1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:15.668199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:15.691995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bda0ba-272d-477d-9519-765316007ecf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:15.700920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615d8d-1800-4b1f-b8d1-1f3abf9c66ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:15.701641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615d8d-1800-4b1f-b8d1-1f3abf9c66ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.701983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615d8d-1800-4b1f-b8d1-1f3abf9c66ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.702454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615d8d-1800-4b1f-b8d1-1f3abf9c66ba admin admin] PATH_INFO: `/v3/projects/69bbd9a7e9db4fda8145328616b2d2f1/users/389fb0c4f26142fdbd9d2ae6499be7b1/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.705928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615d8d-1800-4b1f-b8d1-1f3abf9c66ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bbd9a7e9db4fda8145328616b2d2f1, user_id=389fb0c4f26142fdbd9d2ae6499be7b1, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:15.711397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615d8d-1800-4b1f-b8d1-1f3abf9c66ba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:15.735988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1030/2084] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:15.738367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.738768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1055/2085] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:15 2022] PUT /identity/v3/projects/69bbd9a7e9db4fda8145328616b2d2f1/users/389fb0c4f26142fdbd9d2ae6499be7b1/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:15.739731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.740180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.748860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f0fdb-d4f6-47c5-a290-c23b1f060350 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.749413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f0fdb-d4f6-47c5-a290-c23b1f060350 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.749908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f0fdb-d4f6-47c5-a290-c23b1f060350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:15.796275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-308f0fdb-d4f6-47c5-a290-c23b1f060350 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:15.799152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-308f0fdb-d4f6-47c5-a290-c23b1f060350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:15.801210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] MFA Rules not processed for user `5d7447e6c7db45c49ef11f7abfe4dde3`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:15.803109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:15.968203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1056/2086] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:15.976544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:15.976544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:15.976544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.013626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1031/2087] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:16.021060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.035472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] MFA Rules not processed for user `389fb0c4f26142fdbd9d2ae6499be7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:16.037422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:16.053802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6436704-7fb6-4fd8-af21-4432cdc75eb9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.066103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.067332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.067683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.068479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.069285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:16.069803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.072739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.109292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1032/2088] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:16.111022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.116318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:16.151207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9786d-cf8c-4940-bf92-4c225c50e78f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.179190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.183819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.187453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.188472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.192170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.195195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1057/2089] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:16.198976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.207389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.238372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b944c36-d730-48d8-8f25-a5cf95d9f823 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.248164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.249924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.250705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.251326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.253201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.257962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.273056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1033/2090] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:16 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:16.276213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.278052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:16.304592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1058/2091] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:16 2022] POST /identity/v3/projects => generated 361 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:16.306060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.356642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165712-19de-4159-ba70-296b32feb4bb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.364006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:16.388932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:16.415411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c62c3a-b1d5-473c-b093-9911a5eb0802 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.420401 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.421117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.421378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.421624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.422259 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.424173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.444068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.444068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.444556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.444758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.448081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:16.456348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1059/2092] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:16 2022] POST /identity/v3/users => generated 303 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:16.457354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.468711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d272397-7807-48e5-b76a-2ed9f0b9de9b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.474013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.474779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.475188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.475354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.476059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.479773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.509725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1060/2093] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:16 2022] POST /identity/v3/users => generated 281 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:16.513748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.525215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050b2c0-98ad-456b-ad52-c8d509446def admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.530046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.530670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.530962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.531127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.531293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.531859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.532528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.533544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.542283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1034/2094] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:16.543095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.547808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1061/2095] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:16.552239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.556510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b3e28-cdb2-4199-9f5a-3838f699fb14 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.563713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84b4a4-68b5-4e06-a0d1-ac1dccb8ee2b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.564916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.565693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.566112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.566497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.567237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.569110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.569645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.570424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.571331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.571714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] PATH_INFO: `/v3/projects/985669ad91514d5fa8e7f36f0b0410da/users/1c5a18a3a98442eb92e5d1b31ea90981/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.585059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1035/2096] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:16.588364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.600498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985669ad91514d5fa8e7f36f0b0410da, user_id=1c5a18a3a98442eb92e5d1b31ea90981, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.602257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59103ed3-23d5-488f-98d7-3ba45dbbac03 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.603341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.608720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.609555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.609968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.610360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.611103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.613107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.620263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1062/2097] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:16 2022] PUT /identity/v3/projects/985669ad91514d5fa8e7f36f0b0410da/users/1c5a18a3a98442eb92e5d1b31ea90981/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:16.624339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.626061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1036/2098] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:16.635283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.646312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605df553-25a5-4070-83bd-3ab9ea2b0a1d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.725633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585bc6a-f315-40d8-95cd-09be29402684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.726325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585bc6a-f315-40d8-95cd-09be29402684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.726602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585bc6a-f315-40d8-95cd-09be29402684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.726889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585bc6a-f315-40d8-95cd-09be29402684 admin admin] PATH_INFO: `/v3/projects/75f04cac82a34415a2cf4d6a9ceaf102/users/2092130bf0e14f4ab26840c8c57966bb/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.746844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a32038-de14-433a-9dde-59b36a7dac30 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.765318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.765435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585bc6a-f315-40d8-95cd-09be29402684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f04cac82a34415a2cf4d6a9ceaf102, user_id=2092130bf0e14f4ab26840c8c57966bb, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.765531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.765531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.765531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.765531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.765531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.765931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585bc6a-f315-40d8-95cd-09be29402684 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.779885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1063/2099] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:16.781116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:16.792760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e46c-a0c5-4b3e-825b-0639c901997b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:16.797495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1037/2100] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:16 2022] PUT /identity/v3/projects/75f04cac82a34415a2cf4d6a9ceaf102/users/2092130bf0e14f4ab26840c8c57966bb/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:16.810072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.810349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.810622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.875308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] MFA Rules not processed for user `2092130bf0e14f4ab26840c8c57966bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:16.878384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:16.889705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069298ab-9a1b-4852-a3de-fad00b40b497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:16.890488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069298ab-9a1b-4852-a3de-fad00b40b497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.890772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069298ab-9a1b-4852-a3de-fad00b40b497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.891111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069298ab-9a1b-4852-a3de-fad00b40b497 admin admin] PATH_INFO: `/v3/projects/985669ad91514d5fa8e7f36f0b0410da/users/1c5a18a3a98442eb92e5d1b31ea90981/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.893533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069298ab-9a1b-4852-a3de-fad00b40b497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985669ad91514d5fa8e7f36f0b0410da, user_id=1c5a18a3a98442eb92e5d1b31ea90981, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:16.893752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:16.896806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069298ab-9a1b-4852-a3de-fad00b40b497 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:16.925059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1064/2101] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:16 2022] PUT /identity/v3/projects/985669ad91514d5fa8e7f36f0b0410da/users/1c5a18a3a98442eb92e5d1b31ea90981/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:16.951518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:16.953420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:16.953709 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:16.998298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1038/2102] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:17.032307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:17.039205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] MFA Rules not processed for user `1c5a18a3a98442eb92e5d1b31ea90981`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:17.041475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:17.069858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:17.072677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:17.190828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1dee2-4bf1-49bb-ba4e-4d0e521402cb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:17.266499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1065/2103] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:17.300089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:17.300801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:17.301282 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:17.301533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:17.329720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:17.341852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f9885-39eb-4a42-8022-bd63c16c476d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:17.346238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:17.346863 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:17.347128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:17.347404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:17.358728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:17.360399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:17.386126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1066/2104] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:17.443077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:17.445050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:17.446877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:17.461148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1039/2105] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:17.482436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:17.483235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737db6d-8088-42c6-afc1-ea40ebe47f35 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:17.488180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dbdc3-c0c4-4f52-bf91-8b59603d77c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:17.489268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbdc3-c0c4-4f52-bf91-8b59603d77c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:17.489723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbdc3-c0c4-4f52-bf91-8b59603d77c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:17.490001 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbdc3-c0c4-4f52-bf91-8b59603d77c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:17.531526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:17.601381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dbdc3-c0c4-4f52-bf91-8b59603d77c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:17.603548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dbdc3-c0c4-4f52-bf91-8b59603d77c6 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:17.605851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb414d-0649-483c-839d-56306fa4847f service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:17.613035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:17.613680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:17.615454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:17.615454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] PATH_INFO: `/v3/users/2092130bf0e14f4ab26840c8c57966bb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:17.615454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2092130bf0e14f4ab26840c8c57966bb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:17.617332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1067/2106] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:17.618079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:17.679826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2092130bf0e14f4ab26840c8c57966bb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:17.688949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2092130bf0e14f4ab26840c8c57966bb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:17.689685 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2092130bf0e14f4ab26840c8c57966bb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:17.690894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2092130bf0e14f4ab26840c8c57966bb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:17.700851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2092130bf0e14f4ab26840c8c57966bb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:17.726263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1040/2107] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:17 2022] DELETE /identity/v3/users/2092130bf0e14f4ab26840c8c57966bb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:17.733981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:17.734274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:17.734604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:17.755264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:17.756507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:17.841076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1068/2108] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:17.851360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:17.853660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:17.957645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7415e0f-2836-4e03-a573-1809dab74cfc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:17.964019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3092609-7c08-4818-9e73-d7304e24e56b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:17.964916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092609-7c08-4818-9e73-d7304e24e56b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:17.965397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092609-7c08-4818-9e73-d7304e24e56b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:17.965776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092609-7c08-4818-9e73-d7304e24e56b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:17.980379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3092609-7c08-4818-9e73-d7304e24e56b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:17.982092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3092609-7c08-4818-9e73-d7304e24e56b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:18.000040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1041/2109] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:18.597955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:18.600934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:18.621805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759eb1a-c41c-41f6-b636-228becb15701 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:18.631014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:18.632688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:18.634017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:18.649246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] PATH_INFO: `/v3/projects/75f04cac82a34415a2cf4d6a9ceaf102` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:18.651843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f04cac82a34415a2cf4d6a9ceaf102)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:18.671810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:18.766400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:18.806530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f04cac82a34415a2cf4d6a9ceaf102', 'request_id': 'req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:18.814723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f04cac82a34415a2cf4d6a9ceaf102', 'request_id': 'req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:18.825356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f04cac82a34415a2cf4d6a9ceaf102', 'request_id': 'req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:18.837711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] The token cache is being invalidate because project 75f04cac82a34415a2cf4d6a9ceaf102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:18.838077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:18.838656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0b8bd116-97c7-44ff-aff6-f4b474d3a28b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:18.840448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1069/2110] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:18 2022] DELETE /identity/v3/projects/75f04cac82a34415a2cf4d6a9ceaf102 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:18.845597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51063731-0c11-4160-97da-2a423c390d71 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:18.846224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51063731-0c11-4160-97da-2a423c390d71 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:18.846224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51063731-0c11-4160-97da-2a423c390d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:18.866636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-51063731-0c11-4160-97da-2a423c390d71 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:18.867820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-51063731-0c11-4160-97da-2a423c390d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:18.917290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:18.941731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1042/2111] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:18.950500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:18.950903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:18.951217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:18.979275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:18.981573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:18.994235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1070/2112] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:18.997603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51063731-0c11-4160-97da-2a423c390d71 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.008046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51063731-0c11-4160-97da-2a423c390d71 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.014961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.015644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.016059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.016400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.017401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:19.017986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.020238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.044609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1043/2113] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:19.049705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.060840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041661-0c69-43f9-9406-7c716b0679cf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.076022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.076022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.076022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.076022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.076702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.078528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.113303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1071/2114] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:19 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:19.119461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.131639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33590b-9b8e-47f8-884a-527f148a0391 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.217316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.218198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.218502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.218807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.219453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.221897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.248864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1044/2115] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:19 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:19.255460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.270606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b58d-725d-48a8-a43f-e1a366762d50 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.277523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.278606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.279110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.279627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.280812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.284192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.302070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1072/2116] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:19.309041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.327244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d0f5b-5d3f-41fa-b879-c9b5a6c084c6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.332417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e335e21-62b1-458d-bd07-91d98fe76b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.333326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e335e21-62b1-458d-bd07-91d98fe76b94 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.333699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e335e21-62b1-458d-bd07-91d98fe76b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.334044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e335e21-62b1-458d-bd07-91d98fe76b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.334770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e335e21-62b1-458d-bd07-91d98fe76b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.336472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e335e21-62b1-458d-bd07-91d98fe76b94 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.348874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1045/2117] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:19.353274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.364305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2f0e-6800-48e9-8272-ded1b6e576c0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.369889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.370512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.370790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.371066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] PATH_INFO: `/v3/projects/69d89797ea72434794d0639529bf1c18/users/b69c69e93bb54cdcbc2289def72b399b/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.388488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d89797ea72434794d0639529bf1c18, user_id=b69c69e93bb54cdcbc2289def72b399b, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.391289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.408173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1073/2118] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:19 2022] PUT /identity/v3/projects/69d89797ea72434794d0639529bf1c18/users/b69c69e93bb54cdcbc2289def72b399b/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:19.428254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.428783 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.429707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.475221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] MFA Rules not processed for user `b69c69e93bb54cdcbc2289def72b399b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:19.476013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:19.563085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1046/2119] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:19.581277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:19.582407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:19.662449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767e45-2fed-43a4-8644-e04152734493 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:19.667407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:19.667884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:19.668191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:19.668664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:19.683120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:19.685153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:19.698531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1074/2120] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:21.782428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:21.785390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:21.859403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:21.861460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:21.874733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d71a5-3003-4b39-943d-1f36d4dc770c service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:21.880814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:21.881677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:21.882139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:21.882526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:21.885476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:21.894153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8638-989f-4094-9b7d-277a8d17bddb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:21.900679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:21.901729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:21.902162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:21.902542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] PATH_INFO: `/v3/users/5d7447e6c7db45c49ef11f7abfe4dde3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:21.904591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d7447e6c7db45c49ef11f7abfe4dde3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:21.908024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:21.962701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:22.010520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:22.013264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fdbe5-b489-4917-86e3-90f72f2b3a85 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:22.021274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d7447e6c7db45c49ef11f7abfe4dde3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:22.028754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1075/2121] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:22.045200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d7447e6c7db45c49ef11f7abfe4dde3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:22.046389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d7447e6c7db45c49ef11f7abfe4dde3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:22.047490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d7447e6c7db45c49ef11f7abfe4dde3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:22.065656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d7447e6c7db45c49ef11f7abfe4dde3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:22.098879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1047/2122] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:21 2022] DELETE /identity/v3/users/5d7447e6c7db45c49ef11f7abfe4dde3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:22.105702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:22.106474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:22.107286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:22.138387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:22.140973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:22.249930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1076/2123] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:22.266532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:22.268972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:22.367916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaab42-aad6-4d19-a65a-452e9be2bcbc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:22.375164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:22.375467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:22.375605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:22.376036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:22.389310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:22.390611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:22.410051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1048/2124] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:22.943512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:22.945115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:22.960528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2915a-afbe-4cee-b37a-a4e743ddde00 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:22.966420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:22.967285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:22.967557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:22.967826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] PATH_INFO: `/v3/projects/9bfe702cbe64436baf89da2e56ba1014` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:22.969080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bfe702cbe64436baf89da2e56ba1014)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:22.971134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:23.033516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bfe702cbe64436baf89da2e56ba1014', 'request_id': 'req-25282b43-9da8-4454-9c73-9dd09e612df4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.041165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bfe702cbe64436baf89da2e56ba1014', 'request_id': 'req-25282b43-9da8-4454-9c73-9dd09e612df4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.050399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bfe702cbe64436baf89da2e56ba1014', 'request_id': 'req-25282b43-9da8-4454-9c73-9dd09e612df4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.063021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] The token cache is being invalidate because project 9bfe702cbe64436baf89da2e56ba1014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:23.063409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.064284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.066008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1077/2125] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:22 2022] DELETE /identity/v3/projects/9bfe702cbe64436baf89da2e56ba1014 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:23.071439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:23.087035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb7f5d-edbb-42b5-87ff-0e6022fa4f11 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:23.150212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:23.150854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:23.151131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:23.151395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] PATH_INFO: `/v3/users/1c5a18a3a98442eb92e5d1b31ea90981` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:23.152550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5a18a3a98442eb92e5d1b31ea90981)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:23.154555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:23.160556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:23.248891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5a18a3a98442eb92e5d1b31ea90981'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.263533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5a18a3a98442eb92e5d1b31ea90981'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.264343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5a18a3a98442eb92e5d1b31ea90981'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.265364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5a18a3a98442eb92e5d1b31ea90981'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.280294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-be25d0c8-46e4-4c3d-a70d-9726eaf9d814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5a18a3a98442eb92e5d1b31ea90981'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.312509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1049/2126] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:23 2022] DELETE /identity/v3/users/1c5a18a3a98442eb92e5d1b31ea90981 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:23.676930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:23.679223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:23.706677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282b43-9da8-4454-9c73-9dd09e612df4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:23.812182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:23.814088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:23.814494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:23.814815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] PATH_INFO: `/v3/projects/985669ad91514d5fa8e7f36f0b0410da` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:23.817670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] RBAC: Authorizing `identity:delete_project(project_id=985669ad91514d5fa8e7f36f0b0410da)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:23.821125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:23.837638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:23.905525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985669ad91514d5fa8e7f36f0b0410da', 'request_id': 'req-7723565a-676d-4913-992f-9495b9bd838c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.916576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985669ad91514d5fa8e7f36f0b0410da', 'request_id': 'req-7723565a-676d-4913-992f-9495b9bd838c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.931156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985669ad91514d5fa8e7f36f0b0410da', 'request_id': 'req-7723565a-676d-4913-992f-9495b9bd838c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.946778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] The token cache is being invalidate because project 985669ad91514d5fa8e7f36f0b0410da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:23.947985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.948568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7723565a-676d-4913-992f-9495b9bd838c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:23.953129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1078/2127] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:23 2022] DELETE /identity/v3/projects/985669ad91514d5fa8e7f36f0b0410da => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:23.957628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:23.958049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:23.958513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:23.986848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:23.988197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:24.080638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1050/2128] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:24.088781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.089089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.089685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.112042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:24.113248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:24.134442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1079/2129] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:24.135945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.152094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2be05-3cfb-4587-9221-eafc2ba779b9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.161923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.162487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.162766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.163044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.163753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:24.164058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.166334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.184064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1051/2130] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:30:24.188423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.197860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074655e-fb12-4df7-bb81-61a259e9ed2e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.203212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.203952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.204103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.204426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.205030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.207847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.244598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1080/2131] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:24 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:24.248956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.260705 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c73ec-1cb5-4aa5-9f18-22c6a837fd6f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.341431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.342196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.342359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.342666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.343333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.345328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.365452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:24.377959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1052/2132] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:24 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:24.379248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.391853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229aab4a-adc6-4f7d-b6ed-25ac92155182 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.396967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.397609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.397889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.398196 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.399710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.402128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.413923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1081/2133] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:24.418632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.430977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770db3d-fa3f-4288-a171-0ebb41d5f460 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.435743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34e12-7c51-4f1b-90c8-34e3a0ef7746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.436351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34e12-7c51-4f1b-90c8-34e3a0ef7746 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.436632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34e12-7c51-4f1b-90c8-34e3a0ef7746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.436904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34e12-7c51-4f1b-90c8-34e3a0ef7746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.437588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34e12-7c51-4f1b-90c8-34e3a0ef7746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.439236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34e12-7c51-4f1b-90c8-34e3a0ef7746 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.451152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1053/2134] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:24.455697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.469405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47a33-2e92-4586-bbed-3114f3f40a44 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.476263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.477814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.478085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.478364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] PATH_INFO: `/v3/projects/efd2755f98fd420b8a835c858250ab22/users/591eb4b0e1014c8484851ea38e20e2eb/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.495575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd2755f98fd420b8a835c858250ab22, user_id=591eb4b0e1014c8484851ea38e20e2eb, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.498303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.514036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1082/2135] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:24 2022] PUT /identity/v3/projects/efd2755f98fd420b8a835c858250ab22/users/591eb4b0e1014c8484851ea38e20e2eb/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:24.526938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.527249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.527524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.557777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] MFA Rules not processed for user `591eb4b0e1014c8484851ea38e20e2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:24.559305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:24.638421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1054/2136] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:24.643005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.712878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7ef6-442c-43bd-a463-cc795e8762cb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.718200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.718832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.719113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.719389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.720543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.722006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.757761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1083/2137] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:24 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:24.762233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.774173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f7c6b-b2b2-42e6-82ca-2948f4e0efc3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.840267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.840909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.841218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.841515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.842189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.844209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.871466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1055/2138] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:24 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:24.877180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.892814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ccfa0-e9b8-4650-b5ef-84468d5c3ca8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.900763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.901528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.901964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.902501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.903962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.907237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.931499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1084/2139] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:24.934737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.951003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c57cc6-fdd7-4b1c-a51d-07f4456f207c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:24.958271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bf7a8-3f8c-48cd-b3c0-26512f81d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:24.958973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf7a8-3f8c-48cd-b3c0-26512f81d13d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:24.959338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf7a8-3f8c-48cd-b3c0-26512f81d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:24.959742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf7a8-3f8c-48cd-b3c0-26512f81d13d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:24.960933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bf7a8-3f8c-48cd-b3c0-26512f81d13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:24.964810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bf7a8-3f8c-48cd-b3c0-26512f81d13d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:24.983633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1056/2140] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:24.988458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:24.998109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:25.007146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64585d00-fa66-49ce-9f9f-806009a48ce2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.020098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.021689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.022194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.022707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] PATH_INFO: `/v3/projects/f760bc05a74f487ca11fb6ff053eba72/users/c64ec7e8e924455295f43d710ee0cf41/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.048846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f760bc05a74f487ca11fb6ff053eba72, user_id=c64ec7e8e924455295f43d710ee0cf41, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.052325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.070265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1085/2141] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:24 2022] PUT /identity/v3/projects/f760bc05a74f487ca11fb6ff053eba72/users/c64ec7e8e924455295f43d710ee0cf41/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:25.089280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.089879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.089879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.125852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] MFA Rules not processed for user `c64ec7e8e924455295f43d710ee0cf41`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:25.127469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:25.185575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:25.220014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1057/2142] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:25.224665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.293103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b8593-af43-4878-8e99-47ffd154b0ef admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.298263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.298854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.299134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.299403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.300072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.302072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.333431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1086/2143] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:25 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:25.337727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.348702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb247d-59b6-4bdb-af41-3839fddeab69 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.418868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.419528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.419743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.420018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.420913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.422849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.453527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1058/2144] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:25 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:25.462150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.481442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bfb0e-07b2-43e2-9676-4942822b7e01 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.489551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.490568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.491057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.491376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.493153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.495189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.511560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1087/2145] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:25.516218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.531006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa3160-34db-4c65-ad55-11b1af35fc3b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.540300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.541373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.541836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.542282 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] PATH_INFO: `/v3/projects/d034536af2d842b7984ecb2dd0ec2ba7/users/4617e580bd3449cdb108506da3d3edbc/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.562400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d034536af2d842b7984ecb2dd0ec2ba7, user_id=4617e580bd3449cdb108506da3d3edbc, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.565786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.585398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1059/2146] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:25 2022] PUT /identity/v3/projects/d034536af2d842b7984ecb2dd0ec2ba7/users/4617e580bd3449cdb108506da3d3edbc/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:25.589995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.602759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd5d3a-523a-4058-bc44-b664fbdad854 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.674215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255d0c1-6436-46dc-8f26-210a562f168f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.675298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255d0c1-6436-46dc-8f26-210a562f168f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.675817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255d0c1-6436-46dc-8f26-210a562f168f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.676305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255d0c1-6436-46dc-8f26-210a562f168f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.677816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255d0c1-6436-46dc-8f26-210a562f168f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.681425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255d0c1-6436-46dc-8f26-210a562f168f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.698763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1088/2147] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:25.703982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.723690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20008b59-1bca-4c49-8a11-b280f9aee886 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:25.731208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:25.732797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.732797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.732797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] PATH_INFO: `/v3/projects/d034536af2d842b7984ecb2dd0ec2ba7/users/4617e580bd3449cdb108506da3d3edbc/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.734805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d034536af2d842b7984ecb2dd0ec2ba7, user_id=4617e580bd3449cdb108506da3d3edbc, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:25.737914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:25.751774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1060/2148] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:25 2022] PUT /identity/v3/projects/d034536af2d842b7984ecb2dd0ec2ba7/users/4617e580bd3449cdb108506da3d3edbc/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:25.762878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:25.763171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:25.763478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:25.809520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] MFA Rules not processed for user `4617e580bd3449cdb108506da3d3edbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:25.811041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:25.869802 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:25.933376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1089/2149] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:25.989446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:25.991356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:26.105512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be0756-183f-4d38-b3bf-042ae1651820 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:26.112304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18516c13-13f8-47db-9c6c-32f47162cf83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:26.113306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516c13-13f8-47db-9c6c-32f47162cf83 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:26.113763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516c13-13f8-47db-9c6c-32f47162cf83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:26.114166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516c13-13f8-47db-9c6c-32f47162cf83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:26.138322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18516c13-13f8-47db-9c6c-32f47162cf83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:26.141439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18516c13-13f8-47db-9c6c-32f47162cf83 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:26.155695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1061/2150] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:26.228594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:26.230598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:26.249412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b46ccc-97a7-4750-a9bf-18085f378180 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:26.253672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a2efb-e556-41ff-b79e-d59a55c4c02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:26.254469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a2efb-e556-41ff-b79e-d59a55c4c02e service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:26.255236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a2efb-e556-41ff-b79e-d59a55c4c02e service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:26.255563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a2efb-e556-41ff-b79e-d59a55c4c02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:26.378462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a2efb-e556-41ff-b79e-d59a55c4c02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:26.381610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a2efb-e556-41ff-b79e-d59a55c4c02e service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:26.400438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1090/2151] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:30:31.782932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8544587-7c17-431a-8e9d-d18f831f1f06 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:31.784666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8544587-7c17-431a-8e9d-d18f831f1f06 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:31.785310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8544587-7c17-431a-8e9d-d18f831f1f06 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:31.786906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1062/2152] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:30:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:30:31.836129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1361161-ffef-47c9-9a1f-6447809dc65d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:31.837073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1361161-ffef-47c9-9a1f-6447809dc65d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:31.837522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1361161-ffef-47c9-9a1f-6447809dc65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:31.862625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b1361161-ffef-47c9-9a1f-6447809dc65d None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:31.864172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b1361161-ffef-47c9-9a1f-6447809dc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:31.973026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8544587-7c17-431a-8e9d-d18f831f1f06 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:31.978502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c8544587-7c17-431a-8e9d-d18f831f1f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:31.999755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1091/2153] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:30:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:32.019058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9d2c7-47f9-4947-8870-3a4298bb8619 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:32.019777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9d2c7-47f9-4947-8870-3a4298bb8619 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:32.020079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9d2c7-47f9-4947-8870-3a4298bb8619 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:32.022303 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1092/2154] 10.222.0.85 () {54 vars in 961 bytes} [Wed Nov 2 12:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:30:32.044215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:32.044543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:32.044810 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:32.090356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:32.091092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:32.092631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:32.124622 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1093/2155] 10.222.0.85 () {58 vars in 1058 bytes} [Wed Nov 2 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:32.132675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8544587-7c17-431a-8e9d-d18f831f1f06 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:32.135020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:32.150834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:32.150834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:32.150834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:32.150834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] PATH_INFO: `/v3/users/8285583445bc4964a3b4b66b462d9212` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:32.151439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8285583445bc4964a3b4b66b462d9212)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:32.151586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cfce0-d123-42f8-92c5-de3c9a0777e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:32.151719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:32.157645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:32.158874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:32.159336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:32.161487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:32.180480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:32.186988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:32.205890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1094/2156] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:30:32.282379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8285583445bc4964a3b4b66b462d9212'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.312950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8285583445bc4964a3b4b66b462d9212'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.314185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8285583445bc4964a3b4b66b462d9212'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.315186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8285583445bc4964a3b4b66b462d9212'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.334062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-59e3a044-8fa7-4186-92bf-e75dea524c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8285583445bc4964a3b4b66b462d9212'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.381735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1063/2157] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:31 2022] DELETE /identity/v3/users/8285583445bc4964a3b4b66b462d9212 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:32.658611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:32.678205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6eb8-c3d0-4c55-8a5f-d193a7fc4d56 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:32.781739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:32.796470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:32.797156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:32.797510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:32.797757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] PATH_INFO: `/v3/projects/565eec3408344c2e84f2988fb2230c46` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:32.799014 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] RBAC: Authorizing `identity:delete_project(project_id=565eec3408344c2e84f2988fb2230c46)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:32.801344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:32.868006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565eec3408344c2e84f2988fb2230c46', 'request_id': 'req-62074870-159f-405a-8e71-a54b6ceb777d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.878782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565eec3408344c2e84f2988fb2230c46', 'request_id': 'req-62074870-159f-405a-8e71-a54b6ceb777d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.890332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565eec3408344c2e84f2988fb2230c46', 'request_id': 'req-62074870-159f-405a-8e71-a54b6ceb777d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.903460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] The token cache is being invalidate because project 565eec3408344c2e84f2988fb2230c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:32.903807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.904456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62074870-159f-405a-8e71-a54b6ceb777d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:32.906046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1095/2158] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:32 2022] DELETE /identity/v3/projects/565eec3408344c2e84f2988fb2230c46 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:32.911597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:32.912306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:32.912306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:32.936355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:32.937976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:33.036531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1064/2159] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:33.040268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.040550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.040865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.063466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:33.064754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:33.074413 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1096/2160] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:33.080831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.092846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f1f0c-570d-47c3-a2ec-402c55b986fa None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.097986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.098816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.099227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.099485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.100139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:33.100449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.102734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.118728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1065/2161] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:33.124301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.138351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf19e4-5325-4ead-bedd-e67a3330e49c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.144130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.144753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.145144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.145520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.146050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.148132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.192195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1097/2162] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:33 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:33.198359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.210849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52399-d066-4353-9c91-22f7541cbe02 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.299204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:33.334047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.334980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.335329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.335689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.336519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.338569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.379199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1066/2163] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:33 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:33.385107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.398359 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75aec46-b2ae-460d-a3ab-67207c27926c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.406941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.407730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.408068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.408409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.409432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.411897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.432445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1098/2164] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:33.434627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.449973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255bf6-9ce9-463b-8cbe-2e43c40b25f4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.471961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6ae51-d1d1-4753-ae29-fff6ee91a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.473396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6ae51-d1d1-4753-ae29-fff6ee91a77e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.473508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6ae51-d1d1-4753-ae29-fff6ee91a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.473942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6ae51-d1d1-4753-ae29-fff6ee91a77e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.475149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6ae51-d1d1-4753-ae29-fff6ee91a77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.478205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6ae51-d1d1-4753-ae29-fff6ee91a77e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.499029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1067/2165] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:30:33.503427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.517340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d33778-daf7-40d8-8e34-c300345e1393 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.524686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.525766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.526109 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.527274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] PATH_INFO: `/v3/projects/5bc73b74407f4d6184cbd1330ee89145/users/aacb777ebdf445eca64d1e46c2bdeced/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.548022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc73b74407f4d6184cbd1330ee89145, user_id=aacb777ebdf445eca64d1e46c2bdeced, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.551992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.573290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1099/2166] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:33 2022] PUT /identity/v3/projects/5bc73b74407f4d6184cbd1330ee89145/users/aacb777ebdf445eca64d1e46c2bdeced/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:33.579003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560883-7288-420d-94a4-33f301b69679 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.579347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560883-7288-420d-94a4-33f301b69679 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.579613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560883-7288-420d-94a4-33f301b69679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.601743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a0560883-7288-420d-94a4-33f301b69679 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:33.603127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0560883-7288-420d-94a4-33f301b69679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:33.685205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1068/2167] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:33.706078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:33.707319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:33.831976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45209c-369c-4aff-b068-e058c1d12fdf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:33.836561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:33.837240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:33.837509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:33.837772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:33.848201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:33.850034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:33.862013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1100/2168] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:35.764629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560883-7288-420d-94a4-33f301b69679 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:35.767092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0560883-7288-420d-94a4-33f301b69679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:35.823830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560883-7288-420d-94a4-33f301b69679 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:35.853477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:35.856228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:35.858171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:35.858552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] PATH_INFO: `/v3/users/98f0b63b9c1e4a459bdf8d30e2efc23c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:35.864923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f0b63b9c1e4a459bdf8d30e2efc23c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:35.871616 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:36.135735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f0b63b9c1e4a459bdf8d30e2efc23c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.157440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f0b63b9c1e4a459bdf8d30e2efc23c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.158489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f0b63b9c1e4a459bdf8d30e2efc23c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.160546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f0b63b9c1e4a459bdf8d30e2efc23c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.196228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c0c72d0-252d-4a10-83e9-4c881c776a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f0b63b9c1e4a459bdf8d30e2efc23c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.227191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:36.231267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:36.251480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1069/2169] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:35 2022] DELETE /identity/v3/users/98f0b63b9c1e4a459bdf8d30e2efc23c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:36.256266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:36.256739 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:36.258250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:36.262716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386c595-a9b0-40d9-b01d-2fead37775b5 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:36.322555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:36.326275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:36.332921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:36.483958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:36.485527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:36.486106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:36.486758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] PATH_INFO: `/v3/users/591eb4b0e1014c8484851ea38e20e2eb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:36.489839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591eb4b0e1014c8484851ea38e20e2eb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:36.493777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:36.528672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:36.642905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1070/2170] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:36.668744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:36.670809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:36.699214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591eb4b0e1014c8484851ea38e20e2eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.762652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591eb4b0e1014c8484851ea38e20e2eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.769350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591eb4b0e1014c8484851ea38e20e2eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.772925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591eb4b0e1014c8484851ea38e20e2eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.822041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d05dc41-79b2-480d-a059-5958027ed5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591eb4b0e1014c8484851ea38e20e2eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:36.839866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3c81f-143a-4516-aed3-5e7540a08fd2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:36.874652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:36.874652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:36.874652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:36.877600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:36.888195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:36.920954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1101/2171] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:36 2022] DELETE /identity/v3/users/591eb4b0e1014c8484851ea38e20e2eb => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:36.926437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:36.926736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:36.927005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:36.934579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:36.936517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:36.992905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1071/2172] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:30:37.035992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:37.037678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:37.295558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1102/2173] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:37.338477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:37.340896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:37.395390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab6c18-a505-48ac-a3ff-f7219ffe2fe3 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:37.541556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:37.543388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:37.543714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:37.543990 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:37.558064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:37.559666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:37.575221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1072/2174] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:38.632980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:38.635698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:38.652541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:38.655013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:38.691788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fc735-6fe9-44e3-a50a-45765c3e6292 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:38.692834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f628df-4cc5-4946-950f-4049b122d06d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:38.712503 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:38.719224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:38.719872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:38.720508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:38.721050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:38.721181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:38.721414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] PATH_INFO: `/v3/projects/293a09cd5b4e4d8b9cfa67ba52c9366c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:38.721522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] PATH_INFO: `/v3/projects/efd2755f98fd420b8a835c858250ab22` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:38.728731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd2755f98fd420b8a835c858250ab22)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:38.730416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293a09cd5b4e4d8b9cfa67ba52c9366c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:38.732538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:38.734452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:38.884338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:38.896344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293a09cd5b4e4d8b9cfa67ba52c9366c', 'request_id': 'req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:38.916472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd2755f98fd420b8a835c858250ab22', 'request_id': 'req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:38.923564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293a09cd5b4e4d8b9cfa67ba52c9366c', 'request_id': 'req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:38.926073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd2755f98fd420b8a835c858250ab22', 'request_id': 'req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:38.941156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd2755f98fd420b8a835c858250ab22', 'request_id': 'req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:38.956980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293a09cd5b4e4d8b9cfa67ba52c9366c', 'request_id': 'req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.002654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] The token cache is being invalidate because project 293a09cd5b4e4d8b9cfa67ba52c9366c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:39.002842 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.008781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.010098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] The token cache is being invalidate because project efd2755f98fd420b8a835c858250ab22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:39.013188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.016029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1073/2175] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:38 2022] DELETE /identity/v3/projects/293a09cd5b4e4d8b9cfa67ba52c9366c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:39.016287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.025560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:39.028600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1103/2176] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:38 2022] DELETE /identity/v3/projects/efd2755f98fd420b8a835c858250ab22 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:39.033485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:39.034582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:39.037545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:39.099566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:39.233102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e673188-02e2-4d8d-ab7c-ff64d8c4866b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:39.252979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:39.256044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:39.256765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:39.257436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] PATH_INFO: `/v3/users/c64ec7e8e924455295f43d710ee0cf41` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:39.265627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64ec7e8e924455295f43d710ee0cf41)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:39.270263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:39.348488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e3a61-b1ec-49ee-8b0f-7e427d3e5478 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:39.373386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:39.374187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:39.374651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:39.374780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] PATH_INFO: `/v3/users/389fb0c4f26142fdbd9d2ae6499be7b1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:39.384366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=389fb0c4f26142fdbd9d2ae6499be7b1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:39.389217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:39.514527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64ec7e8e924455295f43d710ee0cf41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.525814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64ec7e8e924455295f43d710ee0cf41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.526917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64ec7e8e924455295f43d710ee0cf41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.528970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64ec7e8e924455295f43d710ee0cf41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.544335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389fb0c4f26142fdbd9d2ae6499be7b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.551517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64ec7e8e924455295f43d710ee0cf41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.559543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389fb0c4f26142fdbd9d2ae6499be7b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.560427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389fb0c4f26142fdbd9d2ae6499be7b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.561911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389fb0c4f26142fdbd9d2ae6499be7b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.584869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389fb0c4f26142fdbd9d2ae6499be7b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:39.633447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1104/2177] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:39 2022] DELETE /identity/v3/users/c64ec7e8e924455295f43d710ee0cf41 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:39.662606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1074/2178] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:39 2022] DELETE /identity/v3/users/389fb0c4f26142fdbd9d2ae6499be7b1 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:40.592364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:40.594058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:40.819854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:40.903774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b14c5-cfba-488a-bcad-0b3bece79fd6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:40.911036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:40.914636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:41.061686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:41.062258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:41.062850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:41.062850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] PATH_INFO: `/v3/projects/f760bc05a74f487ca11fb6ff053eba72` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:41.064619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f760bc05a74f487ca11fb6ff053eba72)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:41.067969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:41.158077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f760bc05a74f487ca11fb6ff053eba72', 'request_id': 'req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.169956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54188e4-14a9-4de7-b5d1-b306efce53ee admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:41.170498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f760bc05a74f487ca11fb6ff053eba72', 'request_id': 'req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.184166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f760bc05a74f487ca11fb6ff053eba72', 'request_id': 'req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.226334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] The token cache is being invalidate because project f760bc05a74f487ca11fb6ff053eba72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:41.226698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.228876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.230888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1105/2179] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:40 2022] DELETE /identity/v3/projects/f760bc05a74f487ca11fb6ff053eba72 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:41.235164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:41.238192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:41.265577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:41.332541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:41.333805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:41.334330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:41.334816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] PATH_INFO: `/v3/projects/69bbd9a7e9db4fda8145328616b2d2f1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:41.350321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69bbd9a7e9db4fda8145328616b2d2f1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:41.353155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:41.418819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f3ebe-fb72-49b5-9be9-e4b929054ea9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:41.431755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:41.432745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:41.435385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:41.435842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] PATH_INFO: `/v3/users/4617e580bd3449cdb108506da3d3edbc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:41.439238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4617e580bd3449cdb108506da3d3edbc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:41.444281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:41.482329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69bbd9a7e9db4fda8145328616b2d2f1', 'request_id': 'req-c4a7e8ca-71a9-483d-b5f3-02227aabad30'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.502067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69bbd9a7e9db4fda8145328616b2d2f1', 'request_id': 'req-c4a7e8ca-71a9-483d-b5f3-02227aabad30'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.523882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:41.529766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69bbd9a7e9db4fda8145328616b2d2f1', 'request_id': 'req-c4a7e8ca-71a9-483d-b5f3-02227aabad30'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.559418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] The token cache is being invalidate because project 69bbd9a7e9db4fda8145328616b2d2f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:41.560090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.562590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c4a7e8ca-71a9-483d-b5f3-02227aabad30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.566021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1075/2180] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:40 2022] DELETE /identity/v3/projects/69bbd9a7e9db4fda8145328616b2d2f1 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:41.572425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32555e-c6d0-4932-b707-ec15f91ebc50 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:41.572771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32555e-c6d0-4932-b707-ec15f91ebc50 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:41.572988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32555e-c6d0-4932-b707-ec15f91ebc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:41.607920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8e32555e-c6d0-4932-b707-ec15f91ebc50 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:41.610855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4617e580bd3449cdb108506da3d3edbc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.611710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8e32555e-c6d0-4932-b707-ec15f91ebc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:41.649368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4617e580bd3449cdb108506da3d3edbc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.652838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4617e580bd3449cdb108506da3d3edbc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.653638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4617e580bd3449cdb108506da3d3edbc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.668760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4617e580bd3449cdb108506da3d3edbc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:41.736679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1106/2181] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:41 2022] DELETE /identity/v3/users/4617e580bd3449cdb108506da3d3edbc => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:41.738838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:41.742318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:41.830186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1076/2182] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:41.840944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:41.841403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:41.841932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:41.874720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:41.878801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:41.950601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486b75-7b0d-4354-9b76-a6af003a6220 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:41.965967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:41.966437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:41.967595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:41.968177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] PATH_INFO: `/v3/users/48ff09b1c6bc4d1588e4300ee547f8ba` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:41.969490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ff09b1c6bc4d1588e4300ee547f8ba)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:41.973024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:42.143216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1077/2183] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:42.143216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:42.144962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ff09b1c6bc4d1588e4300ee547f8ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.167799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ff09b1c6bc4d1588e4300ee547f8ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.168795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ff09b1c6bc4d1588e4300ee547f8ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.170618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ff09b1c6bc4d1588e4300ee547f8ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.179484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef8d61-b301-4c94-83e0-d2f127713952 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:42.198630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b6dda9bc-bbef-4d92-bd79-7408b52125a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ff09b1c6bc4d1588e4300ee547f8ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.205943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:42.206600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:42.206993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:42.207198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:42.207813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:42.208137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:42.211793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:42.214021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:42.306222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1078/2184] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:42.307902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:42.328612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:42.343267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1107/2185] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:41 2022] DELETE /identity/v3/users/48ff09b1c6bc4d1588e4300ee547f8ba => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:42.349147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:42.349745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:42.350201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:42.414193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:42.417793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:42.427993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e918dd-ca40-40b4-8821-2e84008ad08e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:42.443472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:42.547301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1108/2186] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:42.552553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:42.554961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:42.574182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731d0c6-f7c0-4cbb-987a-783b6c746f06 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:42.600458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:42.608105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:42.608105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:42.608105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] PATH_INFO: `/v3/users/b69c69e93bb54cdcbc2289def72b399b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:42.614172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69c69e93bb54cdcbc2289def72b399b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:42.622991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:42.676411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:42.677026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:42.677325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:42.678457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:42.678457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:42.680822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:42.816130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1079/2187] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:42.822162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:42.837858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:42.858559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69c69e93bb54cdcbc2289def72b399b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.880356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69c69e93bb54cdcbc2289def72b399b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.881780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69c69e93bb54cdcbc2289def72b399b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.883003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69c69e93bb54cdcbc2289def72b399b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.901203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69c69e93bb54cdcbc2289def72b399b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:42.963090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1109/2188] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:42 2022] DELETE /identity/v3/users/b69c69e93bb54cdcbc2289def72b399b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:42.966724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:42.972311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:42.998769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee3195-856f-4929-9b62-bc8139dcdf15 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.138550 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.139314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.141373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.141373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.146071 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.149858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0ee3b-79cf-42b1-a6b3-fe15b70e2f95 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.159595 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.161986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.162602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.164626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] PATH_INFO: `/v3/projects/d034536af2d842b7984ecb2dd0ec2ba7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.168966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d034536af2d842b7984ecb2dd0ec2ba7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.173675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.173838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.175287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.192794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1080/2189] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:43.197454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.199507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.221758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea35f4-92bf-4a62-9c3f-cbfab7e88f06 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.231063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55980a-1902-469e-a6c3-8080e460d96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.231697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55980a-1902-469e-a6c3-8080e460d96c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.232275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55980a-1902-469e-a6c3-8080e460d96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.232275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55980a-1902-469e-a6c3-8080e460d96c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.235286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55980a-1902-469e-a6c3-8080e460d96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.237365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55980a-1902-469e-a6c3-8080e460d96c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.290382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1081/2190] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:43 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:43.292011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.292473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.292612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.331556 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:43.334766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.355350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:43.367932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1082/2191] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:43.367932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.401639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58110-ed52-4bec-b07b-1d87b5d646a2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.432891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:43.465968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd034536af2d842b7984ecb2dd0ec2ba7', 'request_id': 'req-cb019897-2656-4918-96d3-5ebe066e9aea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:43.518084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd034536af2d842b7984ecb2dd0ec2ba7', 'request_id': 'req-cb019897-2656-4918-96d3-5ebe066e9aea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:43.541190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd034536af2d842b7984ecb2dd0ec2ba7', 'request_id': 'req-cb019897-2656-4918-96d3-5ebe066e9aea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:43.566805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] The token cache is being invalidate because project d034536af2d842b7984ecb2dd0ec2ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:43.567102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:43.567988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:43.574040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1110/2192] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:42 2022] DELETE /identity/v3/projects/d034536af2d842b7984ecb2dd0ec2ba7 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:43.574040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.574040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.584653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457488bf-fc7c-47af-899e-99b2a87e075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.584865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457488bf-fc7c-47af-899e-99b2a87e075c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.585166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457488bf-fc7c-47af-899e-99b2a87e075c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.585497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457488bf-fc7c-47af-899e-99b2a87e075c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.590544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457488bf-fc7c-47af-899e-99b2a87e075c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.592564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457488bf-fc7c-47af-899e-99b2a87e075c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.611097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1083/2193] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:43.613849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.614141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.614418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.645304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:43.646590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.685257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1084/2194] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:43.686771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.696060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.732594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb019897-2656-4918-96d3-5ebe066e9aea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.744383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f09575-7379-4ee5-bd47-64dfe7ba2d05 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.745854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.747140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.747445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.749422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.752456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.781246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad84020-c993-448b-8c6b-4e5c220f3d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.782284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad84020-c993-448b-8c6b-4e5c220f3d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.782742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad84020-c993-448b-8c6b-4e5c220f3d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.783460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad84020-c993-448b-8c6b-4e5c220f3d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.785332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad84020-c993-448b-8c6b-4e5c220f3d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.787304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.789447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad84020-c993-448b-8c6b-4e5c220f3d5a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.792025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.812434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1085/2195] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:43.814295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.814295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.815306 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.834727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1111/2196] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:43.836135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.851334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:43.852767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:43.858647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722275b-d039-4817-ac27-18a4cef73d1a service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.874049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1086/2197] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:43.878522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b0c0-8510-498a-a1eb-16cf1a00023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.878789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.879996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b0c0-8510-498a-a1eb-16cf1a00023f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.881479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b0c0-8510-498a-a1eb-16cf1a00023f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.883280 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b0c0-8510-498a-a1eb-16cf1a00023f admin admin] PATH_INFO: `/v3/projects/8de43ea57f2b44229359e179314b2d15/users/6c07346fbdc44eda9b057abc0fcd209a/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.896015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799146d-6287-4561-9dfb-e029fbc0c014 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.909740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.910617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.911026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.911397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.913176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:43.913751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.915858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.934170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1087/2198] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:43.938620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:43.955851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b0c0-8510-498a-a1eb-16cf1a00023f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de43ea57f2b44229359e179314b2d15, user_id=6c07346fbdc44eda9b057abc0fcd209a, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.965218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b0c0-8510-498a-a1eb-16cf1a00023f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.972926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e6e29-cb48-41da-a97d-832d25c62444 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:43.982798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:43.984025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:43.984554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:43.985014 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:43.988005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:43.993919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:43.994153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1112/2199] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:43 2022] PUT /identity/v3/projects/8de43ea57f2b44229359e179314b2d15/users/6c07346fbdc44eda9b057abc0fcd209a/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:44.000631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:44.000916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:44.001233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:44.073580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:44.083824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:44.136583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1088/2200] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:43 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:44.139551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:44.413175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1113/2201] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:30:44.440633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:44.677050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:44.693300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac1caa-7c70-4918-a2f4-f31dd96abe2f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:44.764125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596cf3-d22b-4fa7-af19-06ee13a71b92 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:44.766300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa143e30-99ec-4eb1-9844-9a60e0fafdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:44.769443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa143e30-99ec-4eb1-9844-9a60e0fafdc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:44.770666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa143e30-99ec-4eb1-9844-9a60e0fafdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:44.772581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa143e30-99ec-4eb1-9844-9a60e0fafdc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:44.781861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa143e30-99ec-4eb1-9844-9a60e0fafdc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:44.795487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa143e30-99ec-4eb1-9844-9a60e0fafdc4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:44.906816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1089/2202] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:44 2022] POST /identity/v3/users => generated 288 bytes in 769 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:44.911013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:44.913768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:44.917513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:44.979995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] MFA Rules not processed for user `aacb777ebdf445eca64d1e46c2bdeced`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:44.986955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:45.034933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:45.036365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.036852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.037418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.086690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:45.093896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:45.130350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:45.152584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1114/2203] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:45.154008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:45.198346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129406a-7ca6-4b9b-9aaf-0050c80b563d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:45.221490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:45.222475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.222938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.223392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.224573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:45.237434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:45.252598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1090/2204] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:45.280517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:45.282109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:45.291618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1115/2205] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:45.306379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:45.309693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:45.370824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cab2-d255-43ef-a880-149e44f33d88 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:45.392156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:45.393453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.394018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.394459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] PATH_INFO: `/v3/projects/5a67532f113c4416a39830450683fb61` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.398627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a67532f113c4416a39830450683fb61)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:45.408613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:45.441767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e2e3f-19f5-4216-8547-9e2ca6300d8b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:45.450049 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:45.450872 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.451156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.451425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.473537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:45.475787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:45.489769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1091/2206] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:30:45.490676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:45.505210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7523e80-a875-4c96-a4af-89f1d890b588 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:45.515826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:45.517308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.517626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.517899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.518596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:45.520724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:45.535543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1092/2207] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:45.541528 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:45.560575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b188bc-7ab1-4cce-9cb4-d564fa16f416 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:45.572347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a67532f113c4416a39830450683fb61', 'request_id': 'req-ade760b4-b273-4d03-8819-98f9eca33b14'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:45.590308 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a67532f113c4416a39830450683fb61', 'request_id': 'req-ade760b4-b273-4d03-8819-98f9eca33b14'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:45.608742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a67532f113c4416a39830450683fb61', 'request_id': 'req-ade760b4-b273-4d03-8819-98f9eca33b14'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:45.640739 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] The token cache is being invalidate because project 5a67532f113c4416a39830450683fb61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:45.641220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:45.645887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ade760b4-b273-4d03-8819-98f9eca33b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:45.651649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1116/2208] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:45 2022] DELETE /identity/v3/projects/5a67532f113c4416a39830450683fb61 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:45.653844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58e198-002a-469c-9b97-a59ecb5b1352 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.655119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58e198-002a-469c-9b97-a59ecb5b1352 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.657731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58e198-002a-469c-9b97-a59ecb5b1352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.705581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6f58e198-002a-469c-9b97-a59ecb5b1352 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:45.710605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f58e198-002a-469c-9b97-a59ecb5b1352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:45.814486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:45.815480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.816268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.816268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] PATH_INFO: `/v3/projects/bc3d462f589d4b52b7b2b72db0a2983c/users/1efafaf9f0ca4d18ae568de115c0b665/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:45.841874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3d462f589d4b52b7b2b72db0a2983c, user_id=1efafaf9f0ca4d18ae568de115c0b665, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:45.845015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:45.864585 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1093/2209] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:45 2022] PUT /identity/v3/projects/bc3d462f589d4b52b7b2b72db0a2983c/users/1efafaf9f0ca4d18ae568de115c0b665/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:45.866765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:45.868901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:45.936077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:45.942673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1117/2210] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:45.943944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8eeb6-4c7e-4ad4-8767-989dc4090ae3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:45.944389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8eeb6-4c7e-4ad4-8767-989dc4090ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:45.944768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8eeb6-4c7e-4ad4-8767-989dc4090ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.002352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:46.018418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c3c8eeb6-4c7e-4ad4-8767-989dc4090ae3 None None] MFA Rules not processed for user `1efafaf9f0ca4d18ae568de115c0b665`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:46.020771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c3c8eeb6-4c7e-4ad4-8767-989dc4090ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:46.068180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b1ce-6c3a-492d-81c6-ed7add3d5d5f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:46.087089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:46.088175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.088597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.089191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] PATH_INFO: `/v3/projects/69d89797ea72434794d0639529bf1c18` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.091332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d89797ea72434794d0639529bf1c18)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:46.094390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:46.293727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d89797ea72434794d0639529bf1c18', 'request_id': 'req-20ab5d69-4186-463e-9b02-1cea1306a46e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:46.311502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d89797ea72434794d0639529bf1c18', 'request_id': 'req-20ab5d69-4186-463e-9b02-1cea1306a46e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:46.322447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1118/2211] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:46.324414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8671b-db17-4f81-8826-4dac06a75758 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.324927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8671b-db17-4f81-8826-4dac06a75758 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.333144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8671b-db17-4f81-8826-4dac06a75758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.335615 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d89797ea72434794d0639529bf1c18', 'request_id': 'req-20ab5d69-4186-463e-9b02-1cea1306a46e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:46.363381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] The token cache is being invalidate because project 69d89797ea72434794d0639529bf1c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:46.363710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:46.364549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:46.367294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1094/2212] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:45 2022] DELETE /identity/v3/projects/69d89797ea72434794d0639529bf1c18 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:46.368711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:46.375825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:46.417949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00c8671b-db17-4f81-8826-4dac06a75758 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:46.422023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00c8671b-db17-4f81-8826-4dac06a75758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:46.549514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab5d69-4186-463e-9b02-1cea1306a46e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:46.570284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:46.571336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.571630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.574734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.576667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:46.580185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:46.624874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1119/2213] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:46.626780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.627233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.627670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.641466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1095/2214] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:46 2022] POST /identity/v3/projects => generated 375 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:46.644245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:46.659399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:46.661181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:46.727968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:46.793131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba82372-8fe8-441b-8a1e-b79ce683727a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:46.816202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:46.817195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.817482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.817755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.818438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:46.818748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:46.820454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2853b-fde1-4a78-a26e-3f3beca949f6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:46.833222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1120/2215] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:46.834238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:46.868684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1096/2216] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:46.874073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.875197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.875536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.877263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93cef5-945d-4b3a-9801-622336147477 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:46.891871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:46.895929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.896582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.898461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.899438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:46.904638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:46.907428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:46.909678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:46.929557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1097/2217] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:46.929557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:46.954448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df6b3d-680c-41c5-8dc5-64ac710edc1c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:46.956576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1121/2218] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:46 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:46.960349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:46.962723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:46.963621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.964052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.964606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.965452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:46.967626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:46.977118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fad8a-4edd-40a9-b4ba-ca14099aafba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:46.984367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:46.985332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:46.986326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:46.986746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:46.987592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:30:46.988005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:46.991002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.033191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1122/2219] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:30:47.033191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.037102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1098/2220] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:46 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:47.046286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.068841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61833afe-7f10-4afd-8f62-3e043c21b9f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.105947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:47.265197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d4ce4-251a-42e7-a2ec-36d02f3f4da8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.274394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.275585 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.276085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.276692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.277569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.277995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.278210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.278586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.278740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.279460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.284030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.289193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.314523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1123/2221] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:47.315565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.335411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a021ed-2519-4533-9bf4-ee82fa2de02a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.341566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1f3f-8aae-4671-8d6f-4fe35ca37767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.342350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1f3f-8aae-4671-8d6f-4fe35ca37767 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.342725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1f3f-8aae-4671-8d6f-4fe35ca37767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.343096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1f3f-8aae-4671-8d6f-4fe35ca37767 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.344190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1f3f-8aae-4671-8d6f-4fe35ca37767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.346395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1f3f-8aae-4671-8d6f-4fe35ca37767 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.378700 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1124/2222] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:47 2022] POST /identity/v3/users => generated 289 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:47.382920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.383501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.384008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.424184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1099/2223] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:47 2022] POST /identity/v3/projects => generated 371 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:47.426448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.430988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] MFA Rules not processed for user `6c07346fbdc44eda9b057abc0fcd209a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:47.432467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:47.593128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1125/2224] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:47.595968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.615829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6601a-7600-4674-9f5a-8ebbea2ab056 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.623056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.623796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.624077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.624350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] PATH_INFO: `/v3/projects/a059edd877154e4888076ade384ad786/users/324eb6a530a74ad9b3a92e4555d69baf/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.646610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a059edd877154e4888076ade384ad786, user_id=324eb6a530a74ad9b3a92e4555d69baf, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.650378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.670313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1100/2225] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:47 2022] PUT /identity/v3/projects/a059edd877154e4888076ade384ad786/users/324eb6a530a74ad9b3a92e4555d69baf/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:47.671357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.714450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:47.771400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b548c26-5e85-4ddb-a03e-89de748558ac None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.784842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.786465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.786630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622f49-e094-48f0-8e65-b8e0a3ae979d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.787336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.788226 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.789742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.792586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.793320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.793603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.793924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.794524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.794707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.798061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.824158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1126/2226] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:47.827408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.831192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:47.841205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1101/2227] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:47 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:47.841205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.851696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179587f-4217-4e26-adc3-f965c18c5c40 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.861769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.862559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.862856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.863124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.864732 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.865885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.890904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1102/2228] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:47.891852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.904095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777e23-12ad-4bd1-9528-1cb146174f6a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.905044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:47.911104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.911767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.912330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.912742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.914586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.916903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.957876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1103/2229] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:47.959568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:47.980527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d05b7-af23-475f-9650-c5c0c91751ed admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.988114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b157030-65fe-439b-ad87-82ffd99fe1d4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:47.993060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.993387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:47.993677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:47.993802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:47.994652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:47.996370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:47.998378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:47.999554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.000322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.000821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.009029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1104/2230] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:48.013530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:48.034186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bc382-bf1e-45aa-9857-55ace902bd6e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:48.055072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:48.057838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.059651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.060402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] PATH_INFO: `/v3/projects/a059edd877154e4888076ade384ad786/users/324eb6a530a74ad9b3a92e4555d69baf/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.066696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a059edd877154e4888076ade384ad786, user_id=324eb6a530a74ad9b3a92e4555d69baf, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:48.078669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:48.105356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:48.108765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:48.112943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1105/2231] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:48 2022] PUT /identity/v3/projects/a059edd877154e4888076ade384ad786/users/324eb6a530a74ad9b3a92e4555d69baf/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:48.116375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:48.140671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1127/2232] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:48.141721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:48.170888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54690a87-cf43-41cc-a85b-8744f278a10f service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:48.323598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1eba8c-82ef-4d76-9594-9ff4664834eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:48.324241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eba8c-82ef-4d76-9594-9ff4664834eb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.324524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eba8c-82ef-4d76-9594-9ff4664834eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.329957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eba8c-82ef-4d76-9594-9ff4664834eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.330838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eba8c-82ef-4d76-9594-9ff4664834eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:48.334122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eba8c-82ef-4d76-9594-9ff4664834eb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:48.356286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eded0-a8fb-4aa8-9733-822a6561f9fb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:48.367201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:48.368369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.369287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.369876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] PATH_INFO: `/v3/projects/11186aeabdc248989def78417828b0cc/users/3dd0573396af42c18f0e26979bb4113d/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.374665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1128/2233] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:48.382527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d672120-a750-4db3-86dd-6f09ee0a96ed None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.382774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d672120-a750-4db3-86dd-6f09ee0a96ed None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.383086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d672120-a750-4db3-86dd-6f09ee0a96ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.416772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11186aeabdc248989def78417828b0cc, user_id=3dd0573396af42c18f0e26979bb4113d, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:48.425656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:48.456150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6d672120-a750-4db3-86dd-6f09ee0a96ed None None] MFA Rules not processed for user `324eb6a530a74ad9b3a92e4555d69baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:48.461781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d672120-a750-4db3-86dd-6f09ee0a96ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:48.490058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1106/2234] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:48 2022] PUT /identity/v3/projects/11186aeabdc248989def78417828b0cc/users/3dd0573396af42c18f0e26979bb4113d/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:48.513113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:48.664714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:48.672789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:48.722742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f8dad-180f-4b1e-b1e6-9732c0ba1d29 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:48.737254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763109-7f44-40a2-ad85-b0910b417bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:48.737952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763109-7f44-40a2-ad85-b0910b417bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.738227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763109-7f44-40a2-ad85-b0910b417bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.738491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763109-7f44-40a2-ad85-b0910b417bd4 admin admin] PATH_INFO: `/v3/projects/620e39db57f34948b5450e5b90c984cb/users/20a4049d2b2b474a8a830d0702b9b1fa/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.790029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763109-7f44-40a2-ad85-b0910b417bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620e39db57f34948b5450e5b90c984cb, user_id=20a4049d2b2b474a8a830d0702b9b1fa, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:48.792981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763109-7f44-40a2-ad85-b0910b417bd4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:48.805565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1129/2235] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:48.808150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877bf57-2d22-4919-9d01-3150abb22c64 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.809073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877bf57-2d22-4919-9d01-3150abb22c64 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.809477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877bf57-2d22-4919-9d01-3150abb22c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.825601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1107/2236] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:48 2022] PUT /identity/v3/projects/620e39db57f34948b5450e5b90c984cb/users/20a4049d2b2b474a8a830d0702b9b1fa/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:48.828316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:48.833888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:48.834397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:48.869755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] MFA Rules not processed for user `aacb777ebdf445eca64d1e46c2bdeced`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:48.871700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:48.872888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7877bf57-2d22-4919-9d01-3150abb22c64 None None] MFA Rules not processed for user `3dd0573396af42c18f0e26979bb4113d`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:48.876042 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7877bf57-2d22-4919-9d01-3150abb22c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:48.974854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1108/2237] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:48.976871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:49.008079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1130/2238] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:49.008079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:49.008079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:49.008079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:49.035269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:49.036797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:49.141421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988850f0-45d1-4a9e-ae3b-7916df2c9f0f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:49.149770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:49.151118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:49.151699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:49.152250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:49.158507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1131/2239] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:49.159161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:49.159287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:49.159287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:49.199797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc4ba-5b77-4a07-b15a-babef49ac0f1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:49.204452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:49.205281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:49.205632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:49.205974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:49.226203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:49.229234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:49.258262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1132/2240] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:49.259875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:49.262345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:49.310864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:49.313111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8db076-15e8-4b34-9267-06262410620c service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:49.331469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1109/2241] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:49.361304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629015-f245-4d1d-8cc0-7fbbe17bf2e0 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:49.365872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:49.366655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:49.366985 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:49.367243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:49.382135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:49.383597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:49.406538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1133/2242] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:50.023026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc2142-b9c6-43dc-a105-b82ef2769dcf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:50.023517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc2142-b9c6-43dc-a105-b82ef2769dcf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:50.024028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc2142-b9c6-43dc-a105-b82ef2769dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:50.103829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-51cc2142-b9c6-43dc-a105-b82ef2769dcf None None] MFA Rules not processed for user `6c07346fbdc44eda9b057abc0fcd209a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:50.110770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-51cc2142-b9c6-43dc-a105-b82ef2769dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:50.217203 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:50.331898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1110/2243] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:56.749962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:56.780217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46199e2d-c5c7-4cd8-b57c-9db2f105210d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:56.788838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:56.789612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:56.789898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:56.790169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] PATH_INFO: `/v3/users/1efafaf9f0ca4d18ae568de115c0b665` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:56.794320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efafaf9f0ca4d18ae568de115c0b665)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:56.796740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:56.816351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:56.903392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efafaf9f0ca4d18ae568de115c0b665'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:56.924381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efafaf9f0ca4d18ae568de115c0b665'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:56.926705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efafaf9f0ca4d18ae568de115c0b665'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:56.928768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efafaf9f0ca4d18ae568de115c0b665'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:56.941477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efafaf9f0ca4d18ae568de115c0b665'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:57.004816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1134/2244] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:56 2022] DELETE /identity/v3/users/1efafaf9f0ca4d18ae568de115c0b665 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:57.012564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935af654-6a76-4c30-b04d-70ed23068f37 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:57.023822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935af654-6a76-4c30-b04d-70ed23068f37 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:57.024876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935af654-6a76-4c30-b04d-70ed23068f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:57.128235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-935af654-6a76-4c30-b04d-70ed23068f37 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:57.139303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-935af654-6a76-4c30-b04d-70ed23068f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:57.385750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1111/2245] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:57.396719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:57.403349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:57.630348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6e163-7e87-469e-af11-af866e768c4e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:57.647417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:57.649531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:57.650095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:57.651060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:57.679073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:57.680695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:57.708081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1135/2246] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:30:57.993249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:57.994288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:57.994974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.042822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] MFA Rules not processed for user `20a4049d2b2b474a8a830d0702b9b1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:30:58.044697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:58.180324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1112/2247] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:30:58.188339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.190656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:58.205824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de64b6-8c14-48fb-9ae4-548d5fa7a3bf service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.212579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.214605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.215026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.215414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.216176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.218717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.262426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1136/2248] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:30:58 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:30:58.278062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.292635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299d27a-ef6d-426d-b1cd-474de7b28b0a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.317705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.319926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:58.385976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.386913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.387440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.387847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.389468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.393524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.407794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:58.415209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adc666-5186-48b3-a72d-9da3369d2a3c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.427455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.428512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.428940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.429510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] PATH_INFO: `/v3/projects/bc3d462f589d4b52b7b2b72db0a2983c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.431330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3d462f589d4b52b7b2b72db0a2983c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.434876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1113/2249] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:30:58 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:30:58.436467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.438864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.450626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:58.465801 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf13b46-601f-449a-bfaa-794d7cd8fe90 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.488473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.489237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.489574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.489937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.490971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.492885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.513579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3d462f589d4b52b7b2b72db0a2983c', 'request_id': 'req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.519746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1114/2250] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:58.520722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3d462f589d4b52b7b2b72db0a2983c', 'request_id': 'req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.520901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.532441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93467eb-e232-4dc8-9bbf-4707393a6683 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.538653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3d462f589d4b52b7b2b72db0a2983c', 'request_id': 'req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.550251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] The token cache is being invalidate because project bc3d462f589d4b52b7b2b72db0a2983c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:30:58.550524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.551193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.552720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1137/2251] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:58 2022] DELETE /identity/v3/projects/bc3d462f589d4b52b7b2b72db0a2983c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:58.556807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.558455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:58.630945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.632686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.633215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.633656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] PATH_INFO: `/v3/projects/b7b9eca8e3f046fb82ba57f7aca35983/users/1219613bc6c649aca2b85ed5ed05d4fc/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.642562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f5139-07e2-46d1-8526-f5cb9fd55b6f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.648060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.648853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.649234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.649482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] PATH_INFO: `/v3/users/324eb6a530a74ad9b3a92e4555d69baf` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.650756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324eb6a530a74ad9b3a92e4555d69baf)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.652847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.661935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b9eca8e3f046fb82ba57f7aca35983, user_id=1219613bc6c649aca2b85ed5ed05d4fc, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.664707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.689763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1115/2252] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:58 2022] PUT /identity/v3/projects/b7b9eca8e3f046fb82ba57f7aca35983/users/1219613bc6c649aca2b85ed5ed05d4fc/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:58.695019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.697107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:58.722866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324eb6a530a74ad9b3a92e4555d69baf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.733689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324eb6a530a74ad9b3a92e4555d69baf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.734971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324eb6a530a74ad9b3a92e4555d69baf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.736745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324eb6a530a74ad9b3a92e4555d69baf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.753194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324eb6a530a74ad9b3a92e4555d69baf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:30:58.807575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1138/2253] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:30:58 2022] DELETE /identity/v3/users/324eb6a530a74ad9b3a92e4555d69baf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:30:58.854101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76c753-4fbd-4715-9307-2fa81be51e6f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:58.963540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:58.964174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:58.964452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:58.964875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:58.965574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:58.968897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:58.991215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1116/2254] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:30:58.996517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:58.998398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:59.025255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d035d-5ea8-41af-bf35-a5f4e7ebf8e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:59.031779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398a7ce-2cf0-4e77-8e4f-e04f196803b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:59.032781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a7ce-2cf0-4e77-8e4f-e04f196803b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:59.033222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a7ce-2cf0-4e77-8e4f-e04f196803b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:59.034147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a7ce-2cf0-4e77-8e4f-e04f196803b0 admin admin] PATH_INFO: `/v3/projects/b7b9eca8e3f046fb82ba57f7aca35983/users/1219613bc6c649aca2b85ed5ed05d4fc/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:59.038106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398a7ce-2cf0-4e77-8e4f-e04f196803b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b9eca8e3f046fb82ba57f7aca35983, user_id=1219613bc6c649aca2b85ed5ed05d4fc, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:59.066216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398a7ce-2cf0-4e77-8e4f-e04f196803b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:59.081637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:59.098256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1139/2255] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:30:58 2022] PUT /identity/v3/projects/b7b9eca8e3f046fb82ba57f7aca35983/users/1219613bc6c649aca2b85ed5ed05d4fc/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:30:59.762252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:30:59.765109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:30:59.775420 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:30:59.779546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb009-e13b-4d83-b8bb-e13aaab05817 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:30:59.885810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:30:59.887153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:30:59.887816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:30:59.888385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] PATH_INFO: `/v3/projects/a059edd877154e4888076ade384ad786` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:30:59.890423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a059edd877154e4888076ade384ad786)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:30:59.895044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:30:59.997243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a059edd877154e4888076ade384ad786', 'request_id': 'req-ee711ba6-f22a-4627-b045-d49ca5781733'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:00.008107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a059edd877154e4888076ade384ad786', 'request_id': 'req-ee711ba6-f22a-4627-b045-d49ca5781733'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:00.017510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a059edd877154e4888076ade384ad786', 'request_id': 'req-ee711ba6-f22a-4627-b045-d49ca5781733'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:00.045013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] The token cache is being invalidate because project a059edd877154e4888076ade384ad786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:00.045533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:00.047612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ee711ba6-f22a-4627-b045-d49ca5781733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:00.052191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1117/2256] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:30:59 2022] DELETE /identity/v3/projects/a059edd877154e4888076ade384ad786 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:00.057548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.058034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.058119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.082184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:00.086559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:00.208681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1140/2257] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:31:00.216786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.217117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.217445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.239302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:00.241347 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:00.259789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1118/2258] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:00.265715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:00.281913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20290d11-4616-45a5-bef8-5f2dca41738a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:00.289089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:00.301677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.304971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.305436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.308136 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:31:00.308733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:00.310555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:00.353648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1141/2259] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:31:00.357521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:00.369563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6d846-133b-43e9-b366-0cac9d657cbe None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:00.391112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:00.392559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.393101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.393713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.394635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:00.405557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:00.470256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:00.471256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1119/2260] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:00 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:00.488684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa548b3-3087-4c23-b500-425ad5726e99 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:00.629075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:00.630394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.630891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.631676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.632623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:00.636653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:00.695208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1142/2261] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:00 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:00.700322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:00.723668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bc4fd-1be5-44f8-b82e-a4c54c4a0d61 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:00.734116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:00.735447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.736079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.737109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.738647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:00.742408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:00.770177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1120/2262] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:00.773864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:00.804140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffcf63-a9fc-481a-b07d-816063ebb6a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:00.816337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef6fc-c6cc-4e4c-9ff0-d06c19f543c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:00.817762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef6fc-c6cc-4e4c-9ff0-d06c19f543c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.819222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef6fc-c6cc-4e4c-9ff0-d06c19f543c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.819877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef6fc-c6cc-4e4c-9ff0-d06c19f543c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.821472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ef6fc-c6cc-4e4c-9ff0-d06c19f543c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:00.825630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ef6fc-c6cc-4e4c-9ff0-d06c19f543c1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:00.851081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1143/2263] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:00.858028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:00.871827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456230-b8e2-480b-9329-03880ac818b3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:00.878443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:00.879603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.879995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.880383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] PATH_INFO: `/v3/projects/4a9b79a9a3a04cea9255d4ac2dc490ad/users/c5a3336d67674d35868a2438248afc14/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:00.909920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9b79a9a3a04cea9255d4ac2dc490ad, user_id=c5a3336d67674d35868a2438248afc14, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:00.913181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:00.940231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1121/2264] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:00 2022] PUT /identity/v3/projects/4a9b79a9a3a04cea9255d4ac2dc490ad/users/c5a3336d67674d35868a2438248afc14/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:00.952377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:00.952664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:00.953229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:01.066234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] MFA Rules not processed for user `c5a3336d67674d35868a2438248afc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:01.067558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:01.181897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:01.200375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1144/2265] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:01.207983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:01.289837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:01.382057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4dcb8-c2e6-49ce-bfbc-e99e60dd9b24 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:01.395101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:01.395822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:01.396103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:01.396370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:01.397884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:01.399892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:01.451566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1122/2266] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:01 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:01.452878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:01.475319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d498f-94e4-4391-bb48-cb013920cd20 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:01.663260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:01.664362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:01.664611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:01.665115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:01.665887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:01.669478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:01.748903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1145/2267] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:01 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:01.753145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:01.771517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf000395-4e26-4bef-a16a-862bd40da8fc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:01.785794 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:01.786482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:01.788158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:01.788487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:01.790718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:01.797825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:01.833896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1123/2268] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:01.835412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:01.849894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8f9b3-f4ac-461c-94c6-841d7e6adc3d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:01.866051 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c053175-a451-49b3-b44a-d5f55067d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:01.870335 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c053175-a451-49b3-b44a-d5f55067d561 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:01.870811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c053175-a451-49b3-b44a-d5f55067d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:01.871149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c053175-a451-49b3-b44a-d5f55067d561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:01.871949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c053175-a451-49b3-b44a-d5f55067d561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:01.875621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c053175-a451-49b3-b44a-d5f55067d561 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:01.906591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1146/2269] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:01.909629 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:01.940679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa6336-7c20-49ea-94a4-99eabf9f6e5a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:01.954170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:01.954848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:01.955123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:01.955407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] PATH_INFO: `/v3/projects/627977d3d0754db4b99d97a9cd45d3ee/users/6ffa3497d27543379720b7d968221f53/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:01.999435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627977d3d0754db4b99d97a9cd45d3ee, user_id=6ffa3497d27543379720b7d968221f53, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:02.003284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:02.042054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1124/2270] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:01 2022] PUT /identity/v3/projects/627977d3d0754db4b99d97a9cd45d3ee/users/6ffa3497d27543379720b7d968221f53/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:02.159222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:02.159787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:02.159787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:02.224898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] MFA Rules not processed for user `6ffa3497d27543379720b7d968221f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:02.226579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:02.366207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1147/2271] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:02.376465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:02.486047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca297-82dd-48d7-8cbe-0c4e159d9ef4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:02.495464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:02.496096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:02.496383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:02.496795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:02.497733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:02.500279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:02.563529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1125/2272] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:02 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:02.565518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:02.612861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fcfe1-8269-4c1c-9e84-130a7c9a4b97 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:02.763270 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:02.763935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:02.764220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:02.764503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:02.765313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:02.767410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:02.825388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:02.858362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1148/2273] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:02 2022] POST /identity/v3/users => generated 297 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:02.862835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:02.882244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eed56c-0f14-4988-9914-a2228c9f3fbe admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:02.900715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:02.901734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:02.902051 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:02.902330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:02.903085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:02.921698 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:02.965582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1126/2274] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:02.972017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:02.999438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6652a-15e3-44de-9f9b-5544e224c5c3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:03.022453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:03.023363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:03.023832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:03.024288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] PATH_INFO: `/v3/projects/9d87220bac5a4e27a5543af3f70278dc/users/6eb2ed6c17ef4a16aad1943efd7c7886/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:03.110155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d87220bac5a4e27a5543af3f70278dc, user_id=6eb2ed6c17ef4a16aad1943efd7c7886, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:03.119759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:03.181334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:03.183072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1149/2275] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:02 2022] PUT /identity/v3/projects/9d87220bac5a4e27a5543af3f70278dc/users/6eb2ed6c17ef4a16aad1943efd7c7886/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:03.205393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bde08e-6849-492d-86e0-30fae1b5759e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:03.313953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:03.335369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d30e2-0d3d-494d-93df-a7e45a098c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:03.355412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d30e2-0d3d-494d-93df-a7e45a098c0c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:03.356108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d30e2-0d3d-494d-93df-a7e45a098c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:03.356444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d30e2-0d3d-494d-93df-a7e45a098c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:03.357920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d30e2-0d3d-494d-93df-a7e45a098c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:03.360169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d30e2-0d3d-494d-93df-a7e45a098c0c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:03.376986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1127/2276] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:03.383541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:03.401351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec59171-7cdf-491a-b5aa-b9ea9c478c71 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:03.409151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:03.410581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:03.411159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:03.411871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] PATH_INFO: `/v3/projects/9d87220bac5a4e27a5543af3f70278dc/users/6eb2ed6c17ef4a16aad1943efd7c7886/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:03.417349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d87220bac5a4e27a5543af3f70278dc, user_id=6eb2ed6c17ef4a16aad1943efd7c7886, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:03.421429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:03.437170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1150/2277] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:03 2022] PUT /identity/v3/projects/9d87220bac5a4e27a5543af3f70278dc/users/6eb2ed6c17ef4a16aad1943efd7c7886/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:03.462381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:03.462882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:03.463536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:03.555500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] MFA Rules not processed for user `6eb2ed6c17ef4a16aad1943efd7c7886`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:03.557893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:03.801635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1128/2278] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:03.815520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:03.821447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:03.989733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f66a-244d-4964-b2ae-4cf7b8a00410 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:04.000434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:04.009991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:04.010539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:04.010882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:04.194092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:04.197605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:04.210605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1151/2279] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:06.415036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:06.417510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:06.436482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12236b2-e92a-4f74-95d3-6cf052ff0400 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:06.440665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b8e13-6839-4aa2-9a9d-f756473e3ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:06.441328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b8e13-6839-4aa2-9a9d-f756473e3ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:06.441597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b8e13-6839-4aa2-9a9d-f756473e3ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:06.441861 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b8e13-6839-4aa2-9a9d-f756473e3ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:06.531853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b8e13-6839-4aa2-9a9d-f756473e3ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:06.533396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b8e13-6839-4aa2-9a9d-f756473e3ea0 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:06.557144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1129/2280] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:07.655135 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:07.666584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a77b5-9bdf-4bd6-8836-c2e1b63fe205 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:07.671010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:07.671722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:07.672062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:07.672402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:07.691351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:07.694678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:07.723428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1152/2281] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:11.527144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:11.527744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:11.527744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:11.579529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] MFA Rules not processed for user `1219613bc6c649aca2b85ed5ed05d4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:11.581940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:11.749096 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:11.765419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1130/2282] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:11.772053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:11.775493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:11.856605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:11.858766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:11.975010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:12.013439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cda143-4000-47b4-8c6e-6b45596ac597 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:12.019915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:12.020808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:12.021770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:12.022353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:12.024693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:12.055690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4567b81-8680-400a-bb92-ef5f67904d1a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:12.076847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:12.078911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:12.079383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:12.079819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] PATH_INFO: `/v3/users/61f2298168ac47b88c01b11ec1433d75` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:12.086030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f2298168ac47b88c01b11ec1433d75)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:12.097696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:12.151319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:12.153174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e446b8-6c74-4d0c-9d0a-ff52f19af9e8 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:12.171998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1153/2283] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:12.180271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f2298168ac47b88c01b11ec1433d75'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:12.205286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f2298168ac47b88c01b11ec1433d75'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:12.205286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f2298168ac47b88c01b11ec1433d75'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:12.205286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f2298168ac47b88c01b11ec1433d75'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:12.256035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f2298168ac47b88c01b11ec1433d75'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:12.333369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1131/2284] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:11 2022] DELETE /identity/v3/users/61f2298168ac47b88c01b11ec1433d75 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:12.339970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:12.341067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:12.341770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:12.410472 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:12.413594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:12.655683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1154/2285] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:31:12.669682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:12.672162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:12.909043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdd70d-39f7-4b9f-80c5-4246ea02e6f6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:12.926715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:12.928375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:12.928856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:12.929428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:12.950835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:12.957269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:13.022605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1132/2286] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:13.695670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:13.698529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:13.722002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e5da2-908b-4fba-b264-b778efe7ca18 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:13.730443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:13.731331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:13.731726 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:13.732108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] PATH_INFO: `/v3/projects/76325bc9b05344a68aa2e9a8f37bd9ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:13.733628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=76325bc9b05344a68aa2e9a8f37bd9ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:13.735902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:13.810877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:13.812598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:13.828126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76325bc9b05344a68aa2e9a8f37bd9ee', 'request_id': 'req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:13.836423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248399-7d28-4d00-b9e8-4e35a651762e service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:13.837237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76325bc9b05344a68aa2e9a8f37bd9ee', 'request_id': 'req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:13.850200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76325bc9b05344a68aa2e9a8f37bd9ee', 'request_id': 'req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:13.864574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] The token cache is being invalidate because project 76325bc9b05344a68aa2e9a8f37bd9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:13.864961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:13.866058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0aef23b-a0cb-4c2f-b232-d6a2165cb6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:13.868756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1155/2287] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:13 2022] DELETE /identity/v3/projects/76325bc9b05344a68aa2e9a8f37bd9ee => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:13.875581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100421b0-ed35-4590-91b2-6fecd3403382 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:13.876192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100421b0-ed35-4590-91b2-6fecd3403382 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:13.877028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100421b0-ed35-4590-91b2-6fecd3403382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:13.881386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:13.910262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-100421b0-ed35-4590-91b2-6fecd3403382 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:13.913365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-100421b0-ed35-4590-91b2-6fecd3403382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:13.938861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:13.959156 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:13.960191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:13.960703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:13.961243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] PATH_INFO: `/v3/users/c5a3336d67674d35868a2438248afc14` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:13.963024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a3336d67674d35868a2438248afc14)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:13.965945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:14.106530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a3336d67674d35868a2438248afc14'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.114267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1156/2288] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:14.129367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.129914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.130510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.131907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a3336d67674d35868a2438248afc14'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.133614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a3336d67674d35868a2438248afc14'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.135779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a3336d67674d35868a2438248afc14'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.151789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1462b753-ab3c-4713-a721-929f1ea522d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a3336d67674d35868a2438248afc14'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.200346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1133/2289] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:13 2022] DELETE /identity/v3/users/c5a3336d67674d35868a2438248afc14 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:14.207070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.207639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.208418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.210837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:14.218138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:14.269738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:14.272398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:14.387695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1157/2290] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:14.392079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:14.436840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230ec2d-8aba-482e-972c-ccc5acecd9a7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:14.450205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1134/2291] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:14.454035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:14.460671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:14.463069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.465337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.465852 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.468746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:31:14.469320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:14.469965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:14.480008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:14.555425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1158/2292] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:31:14.555425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:14.562772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:14.603549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a6aa-35b5-4b97-9e34-a938cb594642 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:14.617830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:14.618461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.618742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.621692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] PATH_INFO: `/v3/users/3dd0573396af42c18f0e26979bb4113d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.625699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd0573396af42c18f0e26979bb4113d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:14.627667 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:14.640256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:14.669350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32458a7-df9d-4334-85e4-bfda0dcb3ca7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:14.682094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:14.682293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.682488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.682767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.710227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:14.713772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:14.749687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd0573396af42c18f0e26979bb4113d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.753649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1135/2293] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:14.754943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:14.764432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd0573396af42c18f0e26979bb4113d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.765538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd0573396af42c18f0e26979bb4113d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.767204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd0573396af42c18f0e26979bb4113d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.777875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3590dcae-7b72-49f0-a602-1ed0c951cccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd0573396af42c18f0e26979bb4113d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:14.779422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cdf61-2aa7-446f-a006-ca81f2768396 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:14.793733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:14.794397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.794677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.795030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.795663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:14.797650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:14.846088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1159/2294] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:14 2022] DELETE /identity/v3/users/3dd0573396af42c18f0e26979bb4113d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:14.852819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:14.853224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:14.853907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:14.875686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1136/2295] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:14 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:14.882327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:14.884430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:14.918653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:14.920407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:14.968662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:15.047891 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe19eb5-2c68-4e7a-ae3e-a08f2d5af00c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:15.060204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:15.060946 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.061381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.061756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.062514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:15.070142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:15.093978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1160/2296] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:15.116518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:15.117864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1137/2297] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:14 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:31:15.119452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:15.123656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:15.144037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffb5b7-6279-4104-a61a-f9123c80cbd4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:15.163921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:15.165660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.166197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.166706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.167594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:15.169622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:15.192714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1138/2298] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:31:15.197747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:15.219005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24024ba-6c65-4273-80a3-7b47099b74bb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:15.233811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:15.234493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.234918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.235261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.236676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:15.242055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:15.273158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418f3bc-c37d-458e-b093-b4ce2654a0ba None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:15.283412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0edcaf-9e36-4882-92cf-69a5709ef1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:15.284084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0edcaf-9e36-4882-92cf-69a5709ef1ca service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.284350 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0edcaf-9e36-4882-92cf-69a5709ef1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.284654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0edcaf-9e36-4882-92cf-69a5709ef1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.285653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1139/2299] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:31:15.290452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:15.304699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0edcaf-9e36-4882-92cf-69a5709ef1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:15.306307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0edcaf-9e36-4882-92cf-69a5709ef1ca service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:15.322975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc55a-8ee3-4ca4-9b39-2e02dd5caed4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:15.337108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:15.337969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.339186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.340906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] PATH_INFO: `/v3/projects/5a1ad8fc621841d2aa34faa53db6d26b/users/bfa3f71e5c7a4fac903c511407c36bca/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.346509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1161/2300] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:15.379662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1ad8fc621841d2aa34faa53db6d26b, user_id=bfa3f71e5c7a4fac903c511407c36bca, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:15.384008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:15.433771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1140/2301] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:15 2022] PUT /identity/v3/projects/5a1ad8fc621841d2aa34faa53db6d26b/users/bfa3f71e5c7a4fac903c511407c36bca/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:31:15.434987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.435284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.435562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.519325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] MFA Rules not processed for user `bfa3f71e5c7a4fac903c511407c36bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:15.521117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:15.715673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1162/2302] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:31:15.747859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:15.750204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:15.914435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb23151-63e8-4eea-801b-35c63411b4f0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:15.936231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31d15-0862-4023-8006-b535a40accec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:15.936958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31d15-0862-4023-8006-b535a40accec service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:15.939961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31d15-0862-4023-8006-b535a40accec service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:15.940294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31d15-0862-4023-8006-b535a40accec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:15.969367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31d15-0862-4023-8006-b535a40accec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:15.972821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31d15-0862-4023-8006-b535a40accec service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:15.986750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:15.988290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:16.014512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1141/2303] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:16.209742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:16.210180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:16.210583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:16.234780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:16.252170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] MFA Rules not processed for user `20a4049d2b2b474a8a830d0702b9b1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:16.257809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:16.259485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7a00f-443c-4581-a41c-3bf88e9158e2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:16.269601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:16.272649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:16.273417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:16.273838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:16.274210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] PATH_INFO: `/v3/projects/4a9b79a9a3a04cea9255d4ac2dc490ad` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:16.279069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9b79a9a3a04cea9255d4ac2dc490ad)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:16.281360 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:16.420080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1142/2304] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:31:16.421034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:16.421601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9b79a9a3a04cea9255d4ac2dc490ad', 'request_id': 'req-abf43152-fa21-4b8a-9178-6acc697af4a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.430301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:16.438147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9b79a9a3a04cea9255d4ac2dc490ad', 'request_id': 'req-abf43152-fa21-4b8a-9178-6acc697af4a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.453940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9b79a9a3a04cea9255d4ac2dc490ad', 'request_id': 'req-abf43152-fa21-4b8a-9178-6acc697af4a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.472430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5daa33-5af4-43da-9df9-7fb57b1dd927 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:16.479687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] The token cache is being invalidate because project 4a9b79a9a3a04cea9255d4ac2dc490ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:16.480324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.481975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.485281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1163/2305] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:15 2022] DELETE /identity/v3/projects/4a9b79a9a3a04cea9255d4ac2dc490ad => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:31:16.490989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:16.492815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:16.622073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:16.622977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:16.623516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:16.623932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] PATH_INFO: `/v3/projects/11186aeabdc248989def78417828b0cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:16.625492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11186aeabdc248989def78417828b0cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:16.627587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:16.717807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf43152-fa21-4b8a-9178-6acc697af4a5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:16.740848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11186aeabdc248989def78417828b0cc', 'request_id': 'req-4295d70f-b358-45fb-84b3-606124f261a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.756675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11186aeabdc248989def78417828b0cc', 'request_id': 'req-4295d70f-b358-45fb-84b3-606124f261a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.775092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11186aeabdc248989def78417828b0cc', 'request_id': 'req-4295d70f-b358-45fb-84b3-606124f261a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.845709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] The token cache is being invalidate because project 11186aeabdc248989def78417828b0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:16.846222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.854580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4295d70f-b358-45fb-84b3-606124f261a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:16.863171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1143/2306] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:16 2022] DELETE /identity/v3/projects/11186aeabdc248989def78417828b0cc => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:16.877577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c85d28-20c5-4e9d-9944-f0efc209f55f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:16.878135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c85d28-20c5-4e9d-9944-f0efc209f55f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:16.878612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c85d28-20c5-4e9d-9944-f0efc209f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:17.007740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a5c85d28-20c5-4e9d-9944-f0efc209f55f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:17.009571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a5c85d28-20c5-4e9d-9944-f0efc209f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:17.042676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:17.043824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:17.044309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:17.044794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] PATH_INFO: `/v3/users/6ffa3497d27543379720b7d968221f53` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:17.048422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffa3497d27543379720b7d968221f53)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:17.063058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:17.179438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:17.181562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:17.218605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffa3497d27543379720b7d968221f53'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:17.241118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffa3497d27543379720b7d968221f53'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:17.245522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffa3497d27543379720b7d968221f53'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:17.248072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffa3497d27543379720b7d968221f53'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:17.274601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffa3497d27543379720b7d968221f53'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:17.283910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1144/2307] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:17.296069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:17.296511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:17.297174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:17.347444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1164/2308] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:16 2022] DELETE /identity/v3/users/6ffa3497d27543379720b7d968221f53 => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:17.369371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:17.370618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:17.554243 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1145/2309] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:31:17.558332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:17.575369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27dbc2-d8aa-42ac-acfa-b021f9a198e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:17.583824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:17.584461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:17.584742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:17.585052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:17.585924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:31:17.586935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:17.589729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:17.612354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1165/2310] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:31:17.622272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:17.646179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec19f1b-51f5-481e-b0ea-c445b6027bb8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:17.654400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:17.655847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:17.656325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:17.656678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:17.657456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:17.659677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:17.729173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1146/2311] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:17 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:17.734469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:17.758938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34caa7b0-ff9e-4b34-a925-bb3afbd0b14a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:17.966579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:17.967444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:17.967875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:17.968459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:17.970635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:17.976718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.035698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1166/2312] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:17 2022] POST /identity/v3/users => generated 288 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:18.038551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:18.064909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289ac0e-49b7-45c6-b9f3-89cfaeaf6650 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:18.071973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:18.074808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.075492 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.076341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.077277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:18.079648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.093631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:18.098324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:18.137455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1147/2313] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:31:18.138221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86608b35-5aea-4ed0-a5d0-fb249e6145a7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:18.153334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:18.169687 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4435-ad64-4bbf-9201-47d67e2981a0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:18.173949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:18.174636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.174926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.175208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] PATH_INFO: `/v3/projects/627977d3d0754db4b99d97a9cd45d3ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.181639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=627977d3d0754db4b99d97a9cd45d3ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:18.182411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:18.183697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.183942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.184631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.185677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.187254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:18.191571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.223145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1148/2314] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:18.235114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:18.251539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847f378-0887-4986-9eda-f2ba1e9d760c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:18.259579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda5824-cd39-40b0-879a-58c2b179e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:18.260476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda5824-cd39-40b0-879a-58c2b179e2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.261072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda5824-cd39-40b0-879a-58c2b179e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.262454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda5824-cd39-40b0-879a-58c2b179e2d9 admin admin] PATH_INFO: `/v3/projects/386a3f02f91e4741b8962d2497b4fd9b/users/a71ba7ca41504736b944d329ca57b824/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.321848 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda5824-cd39-40b0-879a-58c2b179e2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=386a3f02f91e4741b8962d2497b4fd9b, user_id=a71ba7ca41504736b944d329ca57b824, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:18.329924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda5824-cd39-40b0-879a-58c2b179e2d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.365282 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627977d3d0754db4b99d97a9cd45d3ee', 'request_id': 'req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.377991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1149/2315] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:18 2022] PUT /identity/v3/projects/386a3f02f91e4741b8962d2497b4fd9b/users/a71ba7ca41504736b944d329ca57b824/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:18.380079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.381095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.381538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.397654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627977d3d0754db4b99d97a9cd45d3ee', 'request_id': 'req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.431610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] MFA Rules not processed for user `a71ba7ca41504736b944d329ca57b824`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:18.433033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:18.441787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627977d3d0754db4b99d97a9cd45d3ee', 'request_id': 'req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.486599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] The token cache is being invalidate because project 627977d3d0754db4b99d97a9cd45d3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:18.489229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.492844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.496626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1167/2316] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:18 2022] DELETE /identity/v3/projects/627977d3d0754db4b99d97a9cd45d3ee => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:18.501426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:18.503237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:18.522788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:18.588330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1150/2317] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:18.601120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:18.602859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:18.616963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:18.645484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6c96c-d5fb-4ca3-bcd5-738238746f0e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:18.756753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d273b18-e41b-4297-9db3-b7038cf59135 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:18.761252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d89c23-9e9c-4a36-98a7-d37deb78f7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:18.761900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d89c23-9e9c-4a36-98a7-d37deb78f7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.762180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d89c23-9e9c-4a36-98a7-d37deb78f7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.762630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d89c23-9e9c-4a36-98a7-d37deb78f7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.773857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d89c23-9e9c-4a36-98a7-d37deb78f7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:18.775261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d89c23-9e9c-4a36-98a7-d37deb78f7f1 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.788457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1151/2318] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:31:18.826059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:18.826842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:18.828596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:18.829713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] PATH_INFO: `/v3/users/6eb2ed6c17ef4a16aad1943efd7c7886` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:18.832763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb2ed6c17ef4a16aad1943efd7c7886)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:18.840230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:18.969774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb2ed6c17ef4a16aad1943efd7c7886'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.990632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb2ed6c17ef4a16aad1943efd7c7886'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.992524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb2ed6c17ef4a16aad1943efd7c7886'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:18.994069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb2ed6c17ef4a16aad1943efd7c7886'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:19.034376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb2ed6c17ef4a16aad1943efd7c7886'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:19.069829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1168/2319] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:18 2022] DELETE /identity/v3/users/6eb2ed6c17ef4a16aad1943efd7c7886 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:19.389745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:19.391167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:19.391600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:19.422743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:19.424382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:19.593266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1152/2320] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:19.623054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:19.626611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:19.680250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:19.699790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:19.701836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:19.760507 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e62388-ae13-4b51-bb5c-28a4f8c0c3a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:19.792582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:19.793697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:19.797796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:19.798055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] PATH_INFO: `/v3/users/bfa3f71e5c7a4fac903c511407c36bca` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:19.801162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa3f71e5c7a4fac903c511407c36bca)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:19.805243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:19.878124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadad4e5-3acc-48d4-84b3-d0aba8b02694 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:19.888841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:19.904549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:19.905502 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:19.906512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:19.918241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa3f71e5c7a4fac903c511407c36bca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:19.929498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa3f71e5c7a4fac903c511407c36bca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:19.931321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:19.935198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:19.945191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa3f71e5c7a4fac903c511407c36bca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:19.946265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa3f71e5c7a4fac903c511407c36bca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:19.956568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1153/2321] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:31:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:31:19.960988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:19.964915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:19.965479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ffe11acc-8435-49ac-99a5-41941f47d4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa3f71e5c7a4fac903c511407c36bca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:20.006000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990437c6-1537-41b4-8641-238e7463d1be service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:20.012186 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1169/2322] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:19 2022] DELETE /identity/v3/users/bfa3f71e5c7a4fac903c511407c36bca => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:31:20.022283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:20.023178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:20.024168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:20.024627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] PATH_INFO: `/v3/projects/9d87220bac5a4e27a5543af3f70278dc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:20.028517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:20.030514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:20.031210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:20.072278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d87220bac5a4e27a5543af3f70278dc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:20.074247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:20.103264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:20.108272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:20.267789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:20.297149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d87220bac5a4e27a5543af3f70278dc', 'request_id': 'req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:20.317162 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d87220bac5a4e27a5543af3f70278dc', 'request_id': 'req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:20.339325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1170/2323] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:20.347115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d87220bac5a4e27a5543af3f70278dc', 'request_id': 'req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:20.361815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] The token cache is being invalidate because project 9d87220bac5a4e27a5543af3f70278dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:20.362254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:20.363021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:20.367516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:20.373891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:20.385297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1154/2324] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:19 2022] DELETE /identity/v3/projects/9d87220bac5a4e27a5543af3f70278dc => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:20.652119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4806f4-7551-42bb-bf9c-5f0a1cd0ec9d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:20.662977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:20.755904 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:20.757765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:20.758047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:20.758320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:20.777399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:21.055559 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:21.057190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ad4c8-c31c-45e2-8bac-22d4cf1c63fd service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:21.084658 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1171/2325] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:22.443686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:22.445471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:22.463041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef67c5-c78a-4bcb-ab87-40c071cc5b90 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:22.475567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:22.486037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:22.486355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:22.488029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] PATH_INFO: `/v3/projects/5a1ad8fc621841d2aa34faa53db6d26b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:22.489671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1ad8fc621841d2aa34faa53db6d26b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:22.496093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:22.652589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1ad8fc621841d2aa34faa53db6d26b', 'request_id': 'req-1e9800d8-effe-4535-9fcc-a28dddf63538'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:22.671351 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1ad8fc621841d2aa34faa53db6d26b', 'request_id': 'req-1e9800d8-effe-4535-9fcc-a28dddf63538'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:22.686657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1ad8fc621841d2aa34faa53db6d26b', 'request_id': 'req-1e9800d8-effe-4535-9fcc-a28dddf63538'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:22.711130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] The token cache is being invalidate because project 5a1ad8fc621841d2aa34faa53db6d26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:22.713101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:22.714294 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e9800d8-effe-4535-9fcc-a28dddf63538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:22.717367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1155/2326] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:22 2022] DELETE /identity/v3/projects/5a1ad8fc621841d2aa34faa53db6d26b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:22.724180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:22.725221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:22.725479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:22.752885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:22.754436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:22.939137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:22.939417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:22.940143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:22.945983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1172/2327] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:23.009563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:23.010823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:23.030263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1156/2328] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:23.035715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.050035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d4d6e-5052-49bf-bb0a-783d5d8a95d5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.057182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.061407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.061407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.061407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.061781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:31:23.061868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.063758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.093588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1173/2329] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:31:23.097962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.118600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b412d6-c9c4-4b56-8f9f-7bcd462d6915 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.130809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.131961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.135141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.135792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.139648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.158839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.165317 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.167117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:23.191203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abf1d-54b3-4c92-84b5-b98a64b8be22 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.203175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.205152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.207144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.209239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] PATH_INFO: `/v3/users/a71ba7ca41504736b944d329ca57b824` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.211747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71ba7ca41504736b944d329ca57b824)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.216192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.249095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1157/2330] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:23 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:23.249095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.332129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:23.350664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71ba7ca41504736b944d329ca57b824'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:23.362169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71ba7ca41504736b944d329ca57b824'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:23.362940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71ba7ca41504736b944d329ca57b824'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:23.363652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71ba7ca41504736b944d329ca57b824'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:23.385223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbfaaf-bd63-463f-85ac-d96ec38c63c4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.386241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c12be36a-36c8-4abb-81e7-f37540e30149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71ba7ca41504736b944d329ca57b824'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:23.392702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.393538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.397036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.397036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.397036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.409470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.431390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1174/2331] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:23 2022] DELETE /identity/v3/users/a71ba7ca41504736b944d329ca57b824 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:23.436741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.437483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.438777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.459113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1158/2332] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:23 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:23.460114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.462578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:23.473362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:23.475343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:23.496613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f1420-7f7e-40b0-843d-ca2247465743 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.531155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=83,keystone:DELETE=11,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:23.632545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.633211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.633494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.634424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.635222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.639721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.647820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1175/2333] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:23.660697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.664038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1159/2334] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:23.668909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.672861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:23.686437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578c65a-8f25-4602-b681-63e59faa98b2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.713202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.713839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.714119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.714394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.715249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.717625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.751357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1160/2335] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:23.752788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:23.767070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca993bc7-cb88-4cf6-bbd4-489d75746dd5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.774460 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac137e-cec7-408e-acdf-df9e768b9eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.775154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac137e-cec7-408e-acdf-df9e768b9eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.775545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac137e-cec7-408e-acdf-df9e768b9eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.775834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac137e-cec7-408e-acdf-df9e768b9eb1 admin admin] PATH_INFO: `/v3/projects/ec4933b462f2427ab914edfa0a640e79/users/2fd2ac267a0447c7bf3ff0d7b95aae32/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.795048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac137e-cec7-408e-acdf-df9e768b9eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4933b462f2427ab914edfa0a640e79, user_id=2fd2ac267a0447c7bf3ff0d7b95aae32, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:23.800244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac137e-cec7-408e-acdf-df9e768b9eb1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:23.834034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58fb6d-36d7-483a-83f6-49fc910d9e14 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:23.837718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1161/2336] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:23 2022] PUT /identity/v3/projects/ec4933b462f2427ab914edfa0a640e79/users/2fd2ac267a0447c7bf3ff0d7b95aae32/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:23.854949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.855318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.855585 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:23.926067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] MFA Rules not processed for user `2fd2ac267a0447c7bf3ff0d7b95aae32`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:23.928415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:23.978192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:23.979521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:23.979984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:23.980426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:24.089883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:24.138594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1162/2337] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:24.138958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:24.142737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:24.199704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:24.207598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:24.242903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1176/2338] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:24.447657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ced48-026a-4222-a5ac-e5d9b2653ad9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:24.451902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43ed3b-6491-4635-bd50-ccc6aec6581f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:24.452754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43ed3b-6491-4635-bd50-ccc6aec6581f service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:24.453099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43ed3b-6491-4635-bd50-ccc6aec6581f service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:24.453376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43ed3b-6491-4635-bd50-ccc6aec6581f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:24.482774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43ed3b-6491-4635-bd50-ccc6aec6581f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:24.484212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43ed3b-6491-4635-bd50-ccc6aec6581f service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:24.508054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1163/2339] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:24.532264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:24.548723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecb18f-f973-4ae1-b6ff-8b5b4ed76ee5 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:24.569919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:24.570940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:24.571395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:24.571908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] PATH_INFO: `/v3/users/2fd2ac267a0447c7bf3ff0d7b95aae32` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:24.573763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd2ac267a0447c7bf3ff0d7b95aae32)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:24.576246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:24.719927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd2ac267a0447c7bf3ff0d7b95aae32'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:24.731362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd2ac267a0447c7bf3ff0d7b95aae32'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:24.732588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd2ac267a0447c7bf3ff0d7b95aae32'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:24.733618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd2ac267a0447c7bf3ff0d7b95aae32'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:24.749039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd2ac267a0447c7bf3ff0d7b95aae32'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:24.793847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1177/2340] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:31:24 2022] DELETE /identity/v3/users/2fd2ac267a0447c7bf3ff0d7b95aae32 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:24.795262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:24.799045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:24.799676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:24.823829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:24.852806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:24.857580 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:25.017722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1164/2341] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:25.042836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:25.045662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:25.191040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3610e35-8eb2-4d53-9804-89530e91e96b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:25.196125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd576a-8730-4e8a-ae3c-4f94ea663dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:25.196917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd576a-8730-4e8a-ae3c-4f94ea663dcd service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:25.197106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd576a-8730-4e8a-ae3c-4f94ea663dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:25.197499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd576a-8730-4e8a-ae3c-4f94ea663dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:25.205893 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:25.207823 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:25.244930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd576a-8730-4e8a-ae3c-4f94ea663dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:25.246986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287f31-36a2-489a-8ead-f81ecbb5107b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:25.248565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd576a-8730-4e8a-ae3c-4f94ea663dcd service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:25.260336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:25.262506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:25.265197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:25.269062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] PATH_INFO: `/v3/projects/386a3f02f91e4741b8962d2497b4fd9b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:25.272864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=386a3f02f91e4741b8962d2497b4fd9b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:25.288259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:25.290417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1178/2342] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:25.401151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386a3f02f91e4741b8962d2497b4fd9b', 'request_id': 'req-125acf69-6f30-4637-9d12-7d3de895fc7f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:25.411858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386a3f02f91e4741b8962d2497b4fd9b', 'request_id': 'req-125acf69-6f30-4637-9d12-7d3de895fc7f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:25.429188 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386a3f02f91e4741b8962d2497b4fd9b', 'request_id': 'req-125acf69-6f30-4637-9d12-7d3de895fc7f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:25.455691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] The token cache is being invalidate because project 386a3f02f91e4741b8962d2497b4fd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:25.457079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:25.459474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-125acf69-6f30-4637-9d12-7d3de895fc7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:25.462044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1165/2343] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:25 2022] DELETE /identity/v3/projects/386a3f02f91e4741b8962d2497b4fd9b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:25.467514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:25.468066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:25.468650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:25.537978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:25.542223 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:25.641982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:25.747916 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1179/2344] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:25.757038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:25.757630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:25.757887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:25.803416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:25.812137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:25.833135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1166/2345] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:25.837423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:25.852198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61becb-326c-4743-be00-5c4f79ba9243 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:25.890868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:25.892468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:25.892757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:25.893058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:25.893788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:31:25.894102 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:25.896512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:25.916432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1180/2346] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:31:25.920761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:25.951104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b7dab-6658-4534-bfe1-303ae3b8a9b5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:25.961792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:25.965969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:25.965969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:25.965969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:25.965969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:25.965969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:26.054901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:26.057693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:26.079655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1167/2347] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:31:25 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:26.082511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:26.089218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a7325-4184-4236-9527-42a85f4c9077 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:26.250488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:26.252410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:26.252836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:26.253354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] PATH_INFO: `/v3/projects/ec4933b462f2427ab914edfa0a640e79` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:26.267512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4933b462f2427ab914edfa0a640e79)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:26.273193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:26.322960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774548c-31b9-405d-9966-ab856e5a3441 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:26.352164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:26.353383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:26.355960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:26.357361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:26.358921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:26.362774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:26.384911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:26.450977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1168/2348] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:31:26 2022] POST /identity/v3/users => generated 292 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:31:26.455850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:26.469648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34be3f2-de0c-4ec6-a8af-ce9f27ef0548 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:26.481686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4933b462f2427ab914edfa0a640e79', 'request_id': 'req-4ac05be6-209b-41cb-848f-dfa3164fb561'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:26.494217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4933b462f2427ab914edfa0a640e79', 'request_id': 'req-4ac05be6-209b-41cb-848f-dfa3164fb561'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:26.517679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4933b462f2427ab914edfa0a640e79', 'request_id': 'req-4ac05be6-209b-41cb-848f-dfa3164fb561'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:26.533420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] The token cache is being invalidate because project ec4933b462f2427ab914edfa0a640e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:31:26.534121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:26.535365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:31:26.537951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1181/2349] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:31:26 2022] DELETE /identity/v3/projects/ec4933b462f2427ab914edfa0a640e79 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:26.574785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:26.576781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:26.576781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:26.576781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:26.577334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:26.579039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:26.632898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1169/2350] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:26.640761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:26.643889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:26.694776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac05be6-209b-41cb-848f-dfa3164fb561 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:26.712745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60c39d-3fa9-4025-99b4-242d41f967f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:26.716229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60c39d-3fa9-4025-99b4-242d41f967f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:26.716686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60c39d-3fa9-4025-99b4-242d41f967f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:26.717123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60c39d-3fa9-4025-99b4-242d41f967f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:26.721603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60c39d-3fa9-4025-99b4-242d41f967f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:26.727488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60c39d-3fa9-4025-99b4-242d41f967f7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:26.811429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1182/2351] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:31:26.825157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:26.871733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3094a7-64d3-44eb-b123-2c247247e4c6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:26.892573 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:26.895834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:26.896858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:26.897390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] PATH_INFO: `/v3/projects/64af2cf0e0a14b3ea92c2438baeeb900/users/d655fb9352544adf8b3bac0cfa202d9e/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:26.922700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=64af2cf0e0a14b3ea92c2438baeeb900, user_id=d655fb9352544adf8b3bac0cfa202d9e, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:26.926348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:26.955097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1170/2352] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:31:26 2022] PUT /identity/v3/projects/64af2cf0e0a14b3ea92c2438baeeb900/users/d655fb9352544adf8b3bac0cfa202d9e/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:31:26.960917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd511-b1f8-4285-a691-7b6231f31f1b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:26.961475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd511-b1f8-4285-a691-7b6231f31f1b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:26.961929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd511-b1f8-4285-a691-7b6231f31f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:27.007912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0bbcd511-b1f8-4285-a691-7b6231f31f1b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:27.009449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0bbcd511-b1f8-4285-a691-7b6231f31f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:27.112495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:27.141207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1183/2353] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:27.154333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:27.160673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:27.268263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042a3dc-b7ac-4ca3-b83d-351816deb3ee admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:27.274297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf83c3-dc32-4aeb-b870-9473d9dd5e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:27.274917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf83c3-dc32-4aeb-b870-9473d9dd5e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:27.275221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf83c3-dc32-4aeb-b870-9473d9dd5e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:27.275599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf83c3-dc32-4aeb-b870-9473d9dd5e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:27.286412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf83c3-dc32-4aeb-b870-9473d9dd5e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:27.287958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf83c3-dc32-4aeb-b870-9473d9dd5e53 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:27.303994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1171/2354] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:28.123030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:28.123591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:28.124143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:28.176568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] MFA Rules not processed for user `6c07346fbdc44eda9b057abc0fcd209a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:28.177885 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:28.293071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1184/2355] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:28.315816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b72bd-f81a-4387-84db-32672983a2e7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:28.316363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b72bd-f81a-4387-84db-32672983a2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:28.316742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b72bd-f81a-4387-84db-32672983a2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:28.357521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ab1b72bd-f81a-4387-84db-32672983a2e7 None None] MFA Rules not processed for user `d655fb9352544adf8b3bac0cfa202d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:28.359021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab1b72bd-f81a-4387-84db-32672983a2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:28.448968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:28.483299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1172/2356] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:28.490721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:28.492266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:28.558718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d12384-5235-4c06-a68e-570aa5031825 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:28.563057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0bc4-25b4-413f-b71b-e2f0f087dd83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:28.565867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0bc4-25b4-413f-b71b-e2f0f087dd83 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:28.566162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0bc4-25b4-413f-b71b-e2f0f087dd83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:28.566455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0bc4-25b4-413f-b71b-e2f0f087dd83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:28.583380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0bc4-25b4-413f-b71b-e2f0f087dd83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:28.583380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0bc4-25b4-413f-b71b-e2f0f087dd83 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:28.591995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1185/2357] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:30.234743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a12e-a6c0-4534-86bc-c0299324ac1c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:30.235292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a12e-a6c0-4534-86bc-c0299324ac1c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:30.235292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a12e-a6c0-4534-86bc-c0299324ac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:30.263385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f4d5a12e-a6c0-4534-86bc-c0299324ac1c None None] MFA Rules not processed for user `d655fb9352544adf8b3bac0cfa202d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:30.268387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f4d5a12e-a6c0-4534-86bc-c0299324ac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:30.278579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1173/2358] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:38.587256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=57 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:40.256567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:41.236371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084403f5-1ff7-410c-abdd-6dca74e60c61 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:41.237363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084403f5-1ff7-410c-abdd-6dca74e60c61 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:41.237664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084403f5-1ff7-410c-abdd-6dca74e60c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:41.297611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-084403f5-1ff7-410c-abdd-6dca74e60c61 None None] MFA Rules not processed for user `7c5ccd8c99934f60b4ca26c4990b7412`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:41.299991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-084403f5-1ff7-410c-abdd-6dca74e60c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:41.401145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1186/2359] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:43.649157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f23fe-693f-47ab-9b15-44fdfe406ec9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:43.650008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f23fe-693f-47ab-9b15-44fdfe406ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:43.650008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f23fe-693f-47ab-9b15-44fdfe406ec9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:43.653242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1174/2360] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:31:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:31:43.663025 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9b5a2-4d7d-4dd5-9d6d-163350ca7abe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:43.663608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9b5a2-4d7d-4dd5-9d6d-163350ca7abe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:43.663729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9b5a2-4d7d-4dd5-9d6d-163350ca7abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:43.690694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eef9b5a2-4d7d-4dd5-9d6d-163350ca7abe None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:43.693122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eef9b5a2-4d7d-4dd5-9d6d-163350ca7abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:43.793785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1187/2361] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:31:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:43.812348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f23fe-693f-47ab-9b15-44fdfe406ec9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:43.814375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2a3f23fe-693f-47ab-9b15-44fdfe406ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:43.911384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f23fe-693f-47ab-9b15-44fdfe406ec9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:43.916619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe070e5-1e15-4c28-92d3-528bdb13f33d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:43.917464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe070e5-1e15-4c28-92d3-528bdb13f33d service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:43.917748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe070e5-1e15-4c28-92d3-528bdb13f33d service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:43.918019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe070e5-1e15-4c28-92d3-528bdb13f33d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:43.929679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe070e5-1e15-4c28-92d3-528bdb13f33d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:43.931161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe070e5-1e15-4c28-92d3-528bdb13f33d service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:43.943956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1175/2362] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:44.032351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f7ece-0471-426b-9fe0-fb402088ddbd None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:44.032827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f7ece-0471-426b-9fe0-fb402088ddbd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:44.033337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f7ece-0471-426b-9fe0-fb402088ddbd None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:44.036443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1188/2363] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:31:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:31:44.042539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376b791-f656-4899-85f6-d81ee0709402 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:44.042923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376b791-f656-4899-85f6-d81ee0709402 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:44.043092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376b791-f656-4899-85f6-d81ee0709402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:44.064076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2376b791-f656-4899-85f6-d81ee0709402 None None] MFA Rules not processed for user `e102dacae430417aa150ddd53cf34f31`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:44.065234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2376b791-f656-4899-85f6-d81ee0709402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:44.072767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1176/2364] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:44.086140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f7ece-0471-426b-9fe0-fb402088ddbd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:44.088183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-387f7ece-0471-426b-9fe0-fb402088ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:44.104813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f7ece-0471-426b-9fe0-fb402088ddbd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:44.119393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21cb30-3c3e-41de-8ce8-40cd04398d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:44.120984 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21cb30-3c3e-41de-8ce8-40cd04398d1d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:44.121623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21cb30-3c3e-41de-8ce8-40cd04398d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:44.122112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21cb30-3c3e-41de-8ce8-40cd04398d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:44.135673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21cb30-3c3e-41de-8ce8-40cd04398d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:44.137168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21cb30-3c3e-41de-8ce8-40cd04398d1d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:44.146787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1189/2365] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:31:44.313127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d1c81-a242-4ab9-8b92-b26825b582c3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:44.313710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d1c81-a242-4ab9-8b92-b26825b582c3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:44.314359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d1c81-a242-4ab9-8b92-b26825b582c3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:44.316204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1177/2366] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:31:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:31:44.323187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38f89-9b23-40fa-8e7e-446f56974a91 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:44.323565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38f89-9b23-40fa-8e7e-446f56974a91 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:44.323954 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38f89-9b23-40fa-8e7e-446f56974a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:44.350067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2cb38f89-9b23-40fa-8e7e-446f56974a91 None None] MFA Rules not processed for user `c173404c311b43e29193004dc41480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:44.353197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2cb38f89-9b23-40fa-8e7e-446f56974a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:44.364461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1190/2367] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:31:44.379792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d1c81-a242-4ab9-8b92-b26825b582c3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:44.395280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d1c81-a242-4ab9-8b92-b26825b582c3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:44.399509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:44.400360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:44.400778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:44.401178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:44.419070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:44.420779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:44.440436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1178/2368] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:31:54.344032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=65 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:54.433817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=35 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:31:55.580944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:55.580944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:55.580944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:55.623073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:31:55.624348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:55.702678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1191/2369] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Nov 02 12:31:55.718837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:31:55.720693 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:31:55.828400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851539ed-35b3-4de2-9b9b-c4550ae3dbde service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:31:55.833855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d147-7f97-4759-a07f-1b2f653ec6f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:31:55.834501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d147-7f97-4759-a07f-1b2f653ec6f5 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:31:55.834873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d147-7f97-4759-a07f-1b2f653ec6f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:31:55.835184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d147-7f97-4759-a07f-1b2f653ec6f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:31:55.853871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2d147-7f97-4759-a07f-1b2f653ec6f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:31:55.857178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2d147-7f97-4759-a07f-1b2f653ec6f5 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:31:55.875155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1179/2370] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:31:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:32:05.694889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:32:05.867922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=32 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:32:10.445895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:10.455872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a476a4-1b2b-4ab5-8c24-4d99b31d79af None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:10.460202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:10.460811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:10.461166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:10.461437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:10.463167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:10.536775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:10.538265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:10.549173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1192/2371] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:32:10.989727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:10.990677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:10.991127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:11.016025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:11.019310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:11.055429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1180/2372] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:32:11.059734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:11.077246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966aa83-2578-43dd-a2b7-72d2bc68177c service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:11.084362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a31d42-6428-4c9a-a945-089b2d659c23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:11.085283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a31d42-6428-4c9a-a945-089b2d659c23 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:11.085733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a31d42-6428-4c9a-a945-089b2d659c23 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:11.086181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a31d42-6428-4c9a-a945-089b2d659c23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:11.109836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a31d42-6428-4c9a-a945-089b2d659c23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:11.112343 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a31d42-6428-4c9a-a945-089b2d659c23 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:11.125377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1193/2373] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:32:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:32:14.181917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:14.183367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:14.333456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8eac-a80f-45d8-a125-2fadecd994b0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:14.339916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:14.340748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:14.341204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:14.341520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:14.345270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:14.366150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:14.368544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:14.396663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1181/2374] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:32:21.119262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=36 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:32:23.867993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b087-ca37-46a9-9852-30fc4307aef7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:23.868697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b087-ca37-46a9-9852-30fc4307aef7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:23.868697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b087-ca37-46a9-9852-30fc4307aef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:23.895234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3253b087-ca37-46a9-9852-30fc4307aef7 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:23.896544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3253b087-ca37-46a9-9852-30fc4307aef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:23.906520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1194/2375] 10.222.0.85 () {58 vars in 967 bytes} [Wed Nov 2 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:32:23.927522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:23.940962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608641c-0387-49c8-90b9-81fdafdae472 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:23.947112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:23.947837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:23.948200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:23.948544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:23.959457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:23.961254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:23.973141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1182/2376] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:32:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:32:31.051289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74321d2-06e4-4849-8af8-68a1a9b94ea7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:31.051864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74321d2-06e4-4849-8af8-68a1a9b94ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:31.051955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74321d2-06e4-4849-8af8-68a1a9b94ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:31.071513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b74321d2-06e4-4849-8af8-68a1a9b94ea7 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:31.072719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b74321d2-06e4-4849-8af8-68a1a9b94ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:31.080421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1195/2377] 10.222.0.85 () {58 vars in 967 bytes} [Wed Nov 2 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:32:31.094055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:31.106178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea5050-8697-4188-a8ac-abb9d6411a10 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:31.112248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b56771-08f6-4731-b101-f6f6f4b12499 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:31.112899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56771-08f6-4731-b101-f6f6f4b12499 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:31.113213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56771-08f6-4731-b101-f6f6f4b12499 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:31.113480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b56771-08f6-4731-b101-f6f6f4b12499 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:31.122095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b56771-08f6-4731-b101-f6f6f4b12499 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:31.123692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b56771-08f6-4731-b101-f6f6f4b12499 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:31.132481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1183/2378] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:32:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:32:35.305594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a807a1-b24b-4747-ae0e-97801859a283 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:35.306191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a807a1-b24b-4747-ae0e-97801859a283 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:35.306191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a807a1-b24b-4747-ae0e-97801859a283 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:35.307871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1196/2379] 10.222.0.85 () {54 vars in 921 bytes} [Wed Nov 2 12:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 12:32:35.337332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:35.337885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:35.338164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:35.366910 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] MFA Rules not processed for user `1de535a7ce1e475abdb02308cb3326e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:35.368143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:35.379657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a807a1-b24b-4747-ae0e-97801859a283 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:35.380761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1184/2380] 10.222.0.85 () {58 vars in 1018 bytes} [Wed Nov 2 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:32:35.381770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-85a807a1-b24b-4747-ae0e-97801859a283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:35.395617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:35.397048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:35.408384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a807a1-b24b-4747-ae0e-97801859a283 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:35.412362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aa11e-426d-4f34-9eee-2c4a81e8324b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:35.415834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:35.416895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:35.417307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:35.417757 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] PATH_INFO: `/v3/users/6c07346fbdc44eda9b057abc0fcd209a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:35.418227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:35.418920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:35.419264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:35.419600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c07346fbdc44eda9b057abc0fcd209a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:35.419706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:35.423064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:35.430340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:35.431952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:35.442942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1185/2381] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:32:35.500873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c07346fbdc44eda9b057abc0fcd209a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.514573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c07346fbdc44eda9b057abc0fcd209a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.515773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c07346fbdc44eda9b057abc0fcd209a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.516505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c07346fbdc44eda9b057abc0fcd209a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.525080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-83201e1f-962f-40c3-a28b-cb959dcba525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c07346fbdc44eda9b057abc0fcd209a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.557253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1197/2382] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:32:35 2022] DELETE /identity/v3/users/6c07346fbdc44eda9b057abc0fcd209a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:32:35.771512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:35.787425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c8c93-57c5-45aa-9cba-b5bc71aa50eb service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:35.849137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:35.849759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:35.850035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:35.850303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] PATH_INFO: `/v3/projects/8de43ea57f2b44229359e179314b2d15` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:35.851470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de43ea57f2b44229359e179314b2d15)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:35.853417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:35.875145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:32:35.904122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de43ea57f2b44229359e179314b2d15', 'request_id': 'req-ad5bb59a-64d2-4b0c-a35d-056955b755e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.916783 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de43ea57f2b44229359e179314b2d15', 'request_id': 'req-ad5bb59a-64d2-4b0c-a35d-056955b755e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.928133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de43ea57f2b44229359e179314b2d15', 'request_id': 'req-ad5bb59a-64d2-4b0c-a35d-056955b755e1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.943824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] The token cache is being invalidate because project 8de43ea57f2b44229359e179314b2d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:32:35.944245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.944810 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad5bb59a-64d2-4b0c-a35d-056955b755e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:32:35.946689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1186/2383] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:32:35 2022] DELETE /identity/v3/projects/8de43ea57f2b44229359e179314b2d15 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:32:35.951323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:35.951611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:35.951889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:35.973972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:35.975192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:36.036392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1198/2384] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:32:36.044058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.044346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.044623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.065636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:36.066876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:36.075692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1187/2385] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:32:36.082581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.092892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57a2b-b141-4db6-b88c-78f6de5abf2c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.097669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.098271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.098549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.098819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.099494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:32:36.099794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.101498 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.114717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1199/2386] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:32:36.121336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.131963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3fc3-5784-4b2b-9697-590ea2abff0c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.136860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.137499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.137773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.138039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.138675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.140582 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.170593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1188/2387] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:32:36 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:32:36.175070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.185806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e5fc-f635-414a-ad52-165fb2d73d6b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.240950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:32:36.256972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.257610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.257894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.258165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.258812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.260724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.290183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1200/2388] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:32:36 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:32:36.291316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.302459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17e227-fe5d-4ce1-a15c-1e744721a023 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.308455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.309340 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.309707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.310057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.310708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.313327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.326070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1189/2389] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:32:36.329608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.343101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a7ac-bc9d-4ed6-85b3-69fe8c043266 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.348003 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dc427-671c-46c6-863e-eeb73a792817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.348611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dc427-671c-46c6-863e-eeb73a792817 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.348890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dc427-671c-46c6-863e-eeb73a792817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.349187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dc427-671c-46c6-863e-eeb73a792817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.349833 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dc427-671c-46c6-863e-eeb73a792817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.351662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dc427-671c-46c6-863e-eeb73a792817 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.369272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1201/2390] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:32:36.370506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.382380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75388b-99a6-4847-a7b4-741421692091 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.387486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.388151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.388419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.388824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] PATH_INFO: `/v3/projects/7cdeb5a32018400595945a7b56253a9f/users/935494685aa24fa797befc0f42212b73/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.404059 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdeb5a32018400595945a7b56253a9f, user_id=935494685aa24fa797befc0f42212b73, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.406778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.422130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1190/2391] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:32:36 2022] PUT /identity/v3/projects/7cdeb5a32018400595945a7b56253a9f/users/935494685aa24fa797befc0f42212b73/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:32:36.433657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d326e75-253b-4eae-8280-0e42867b4620 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.433923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d326e75-253b-4eae-8280-0e42867b4620 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.434181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d326e75-253b-4eae-8280-0e42867b4620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.469318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7d326e75-253b-4eae-8280-0e42867b4620 None None] MFA Rules not processed for user `935494685aa24fa797befc0f42212b73`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:32:36.469318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7d326e75-253b-4eae-8280-0e42867b4620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:36.543189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1202/2392] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:32:36.556110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:32:36.558367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:32:36.620975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648aba0-63a1-4053-b022-ed374cbdfe4e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:32:36.624735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:32:36.625388 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:32:36.625669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:32:36.625943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:32:36.635785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:32:36.637263 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:32:36.647608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1191/2393] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:32:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:32:46.549427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:32:46.643253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:03.975148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71d9cf-c049-4d03-9cf2-f08f803922e4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:03.976717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71d9cf-c049-4d03-9cf2-f08f803922e4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:03.977024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71d9cf-c049-4d03-9cf2-f08f803922e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:03.998926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2d71d9cf-c049-4d03-9cf2-f08f803922e4 None None] MFA Rules not processed for user `d655fb9352544adf8b3bac0cfa202d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:04.000138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d71d9cf-c049-4d03-9cf2-f08f803922e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:04.057625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1203/2394] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:33:14.050307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:20.807703 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:20.869628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6f9c-7be4-4675-acac-e7cc5e255c79 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:20.875375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:20.876009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:20.876286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:20.876811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] PATH_INFO: `/v3/users/935494685aa24fa797befc0f42212b73` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:20.877981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] RBAC: Authorizing `identity:delete_user(user_id=935494685aa24fa797befc0f42212b73)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:20.880479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:20.928887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935494685aa24fa797befc0f42212b73'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:20.936264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935494685aa24fa797befc0f42212b73'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:20.937105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935494685aa24fa797befc0f42212b73'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:20.937509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935494685aa24fa797befc0f42212b73'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:20.945476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935494685aa24fa797befc0f42212b73'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:20.963238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1192/2395] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:33:20 2022] DELETE /identity/v3/users/935494685aa24fa797befc0f42212b73 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:33:20.967723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:20.968225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:20.968330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:20.987344 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:20.988530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:21.048202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1204/2396] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:21.057917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.059316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:21.120106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5cae-5807-468b-a42b-09cc9fa5fcee admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.124047 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725dfbb-8dac-40f1-ad4b-7d8a62f5e4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.124656 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725dfbb-8dac-40f1-ad4b-7d8a62f5e4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.124936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725dfbb-8dac-40f1-ad4b-7d8a62f5e4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.125240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725dfbb-8dac-40f1-ad4b-7d8a62f5e4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.135591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725dfbb-8dac-40f1-ad4b-7d8a62f5e4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.137063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725dfbb-8dac-40f1-ad4b-7d8a62f5e4c4 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.147579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1193/2397] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:33:21.435567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.437107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:21.451099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bc7cd-88d3-4a38-88b9-0b94b2acf623 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.457035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.457868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.458146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.458479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] PATH_INFO: `/v3/projects/7cdeb5a32018400595945a7b56253a9f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.459869 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cdeb5a32018400595945a7b56253a9f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.462028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.531305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cdeb5a32018400595945a7b56253a9f', 'request_id': 'req-106137ba-2dcc-470b-9b3e-ee24276cbed7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:21.538528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cdeb5a32018400595945a7b56253a9f', 'request_id': 'req-106137ba-2dcc-470b-9b3e-ee24276cbed7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:21.552883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cdeb5a32018400595945a7b56253a9f', 'request_id': 'req-106137ba-2dcc-470b-9b3e-ee24276cbed7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:21.564008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] The token cache is being invalidate because project 7cdeb5a32018400595945a7b56253a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:33:21.564296 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:21.564927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-106137ba-2dcc-470b-9b3e-ee24276cbed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:21.571855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1205/2398] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:33:21 2022] DELETE /identity/v3/projects/7cdeb5a32018400595945a7b56253a9f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:33:21.576694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.576982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.577286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.597440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:21.599157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:21.617176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:21.666668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1194/2399] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:21.674597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.674884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.675156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.695829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:21.697079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:21.705647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1206/2400] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:21.709984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.720253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca775fe1-2629-44e2-92f3-159460771181 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.725907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.726505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.726815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.727079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.727733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:33:21.728034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.729724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.742899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1195/2401] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:33:21.747902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.758079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0982bd-023b-4f90-aae2-15e00ab76eb6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.762880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.763480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.763752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.764020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.764644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.766565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.797790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1207/2402] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:33:21 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:33:21.802600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.813137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06fb58-f68d-4760-9ef2-824f70e20af3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.876254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.877141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.877426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.877704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.878359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.880804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.906451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1196/2403] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:33:21 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:33:21.910955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.922526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33710e01-d420-42e3-87da-af9d86d8b8b2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.928090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.928812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.929160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.929418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.930153 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.931768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.943671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1208/2404] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:33:21.948020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.959308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2992b-59a5-49f1-b56b-6c151d0fdd44 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:21.967474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3f3cb-2a5c-49f2-ab43-409fd63bceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:21.968509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3f3cb-2a5c-49f2-ab43-409fd63bceca admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:21.968903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3f3cb-2a5c-49f2-ab43-409fd63bceca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:21.969279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3f3cb-2a5c-49f2-ab43-409fd63bceca admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:21.969935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3f3cb-2a5c-49f2-ab43-409fd63bceca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:21.971553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3f3cb-2a5c-49f2-ab43-409fd63bceca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:21.983227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1197/2405] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:33:21.987518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:21.998552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766d96-5e62-4b79-9909-98a08a803764 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:22.004677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:22.005392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:22.005665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:22.005929 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] PATH_INFO: `/v3/projects/76e1252062164975b8ac89f6181c4c4d/users/deebae0bfcac4aa894d4c388d9f4f1e3/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:22.021928 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e1252062164975b8ac89f6181c4c4d, user_id=deebae0bfcac4aa894d4c388d9f4f1e3, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:22.024601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:22.044278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1209/2406] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:33:21 2022] PUT /identity/v3/projects/76e1252062164975b8ac89f6181c4c4d/users/deebae0bfcac4aa894d4c388d9f4f1e3/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:33:22.116276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321fc9-3362-4b64-b7df-c911cda2f655 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:22.116491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321fc9-3362-4b64-b7df-c911cda2f655 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:22.116738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321fc9-3362-4b64-b7df-c911cda2f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:22.148398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7e321fc9-3362-4b64-b7df-c911cda2f655 None None] MFA Rules not processed for user `deebae0bfcac4aa894d4c388d9f4f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:22.149683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7e321fc9-3362-4b64-b7df-c911cda2f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:22.226036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1198/2407] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:22.237347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:22.239362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:22.265221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:22.308421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe877462-edcb-485b-b411-9afaf5051adf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:22.312309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae735329-6d0b-4f92-be70-353e8d38ebbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:22.312942 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735329-6d0b-4f92-be70-353e8d38ebbc service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:22.313260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735329-6d0b-4f92-be70-353e8d38ebbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:22.313531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735329-6d0b-4f92-be70-353e8d38ebbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:22.323373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae735329-6d0b-4f92-be70-353e8d38ebbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:22.324831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae735329-6d0b-4f92-be70-353e8d38ebbc service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:22.335966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1210/2408] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:33:23.745712 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2981ad3-f35e-4434-be97-be625f29b796 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:23.746287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2981ad3-f35e-4434-be97-be625f29b796 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:23.746287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2981ad3-f35e-4434-be97-be625f29b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:23.765505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e2981ad3-f35e-4434-be97-be625f29b796 None None] MFA Rules not processed for user `deebae0bfcac4aa894d4c388d9f4f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:23.766722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e2981ad3-f35e-4434-be97-be625f29b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:23.774207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1199/2409] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:32.330414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=17 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:33.759862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:47.864972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:47.865509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:47.865598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:47.888192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] MFA Rules not processed for user `83593f8338c444b7b117eeaf3e4cdcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:47.890113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:47.960779 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1211/2410] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:49.122174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb176a-0557-413d-baf0-2d607833e42d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:49.122756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb176a-0557-413d-baf0-2d607833e42d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:49.122756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb176a-0557-413d-baf0-2d607833e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:49.143609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-38fb176a-0557-413d-baf0-2d607833e42d None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:49.144824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-38fb176a-0557-413d-baf0-2d607833e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:49.211220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1200/2411] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:49.264670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:49.266120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:49.358878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6b030-5e86-451d-99dc-bfe8080016c2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:49.367283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:49.368064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:49.368448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:49.368807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:49.380949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:49.382713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:49.392456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1212/2412] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:33:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:33:50.015621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12add2-e6f9-409c-8620-f6856d48b1d3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:50.015906 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12add2-e6f9-409c-8620-f6856d48b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:50.016180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12add2-e6f9-409c-8620-f6856d48b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:50.037494 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ed12add2-e6f9-409c-8620-f6856d48b1d3 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:50.038651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ed12add2-e6f9-409c-8620-f6856d48b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:50.046221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1201/2413] 10.222.0.85 () {58 vars in 967 bytes} [Wed Nov 2 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:50.059746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:50.071683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f40d3-d9c7-49bc-8de2-c6be4ae3be2a service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:50.077043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:50.077688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:50.077964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:50.078237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:50.087793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:50.089421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:50.098313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1213/2414] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 12:33:50.350738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d79cfb-6beb-4e41-96b4-4566c8d65774 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:50.351238 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d79cfb-6beb-4e41-96b4-4566c8d65774 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:50.351329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d79cfb-6beb-4e41-96b4-4566c8d65774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:50.374028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e9d79cfb-6beb-4e41-96b4-4566c8d65774 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:50.375277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e9d79cfb-6beb-4e41-96b4-4566c8d65774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:50.383544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1202/2415] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:50.401508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:50.413423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9a55-8835-4320-96c4-66c8ad7ebf54 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:50.418950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:50.419645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:50.419920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:50.420215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:50.435765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:50.438797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:50.454841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1214/2416] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:33:50.820442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:50.820964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:50.821397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:50.844450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:50.845721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:50.853642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1203/2417] 10.222.0.85 () {58 vars in 967 bytes} [Wed Nov 2 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:33:50.866851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:50.878236 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e42b6-f571-44f5-9277-03c4ae2e15f2 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:50.883414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacebc8-3a2a-48a6-b26a-598e5891a365 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:50.884063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacebc8-3a2a-48a6-b26a-598e5891a365 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:50.884342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacebc8-3a2a-48a6-b26a-598e5891a365 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:50.884606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacebc8-3a2a-48a6-b26a-598e5891a365 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:50.895141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacebc8-3a2a-48a6-b26a-598e5891a365 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:50.896839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacebc8-3a2a-48a6-b26a-598e5891a365 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:50.906940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1215/2418] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:33:53.184553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:53.313104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b2b95-fc4c-454f-87a8-d65ffa4a4f36 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:53.319786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:53.320241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:53.320516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:53.321221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] PATH_INFO: `/v3/users/deebae0bfcac4aa894d4c388d9f4f1e3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:53.323027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deebae0bfcac4aa894d4c388d9f4f1e3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:53.325266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:53.404767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deebae0bfcac4aa894d4c388d9f4f1e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:53.413619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deebae0bfcac4aa894d4c388d9f4f1e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:53.414353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deebae0bfcac4aa894d4c388d9f4f1e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:53.414965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deebae0bfcac4aa894d4c388d9f4f1e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:53.423223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deebae0bfcac4aa894d4c388d9f4f1e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:53.443830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1204/2419] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:33:53 2022] DELETE /identity/v3/users/deebae0bfcac4aa894d4c388d9f4f1e3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:33:53.448161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:53.448647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:53.448820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:53.472481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:53.475027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:53.531162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:53.545250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1216/2420] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:53.555450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:53.557076 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:53.567785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:53.651123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4bd54-aa55-4f50-b4bf-0d187a12cf44 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:53.655334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b85d3-631f-4d0c-a7e9-40e58ab1e794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:53.656453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b85d3-631f-4d0c-a7e9-40e58ab1e794 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:53.656733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b85d3-631f-4d0c-a7e9-40e58ab1e794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:53.657038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b85d3-631f-4d0c-a7e9-40e58ab1e794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:53.668715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b85d3-631f-4d0c-a7e9-40e58ab1e794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:53.670377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b85d3-631f-4d0c-a7e9-40e58ab1e794 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:53.693118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1205/2421] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:33:53.969704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:53.972571 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:54.002297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84734ec-f4fd-4af4-8ede-448ec26777d0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:54.027566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:54.028594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.029131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.029864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] PATH_INFO: `/v3/projects/76e1252062164975b8ac89f6181c4c4d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.031995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e1252062164975b8ac89f6181c4c4d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:54.035874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:54.128180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e1252062164975b8ac89f6181c4c4d', 'request_id': 'req-08e05b95-8630-4350-bdcc-f1b390bd6ab9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:54.147090 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e1252062164975b8ac89f6181c4c4d', 'request_id': 'req-08e05b95-8630-4350-bdcc-f1b390bd6ab9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:54.165560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e1252062164975b8ac89f6181c4c4d', 'request_id': 'req-08e05b95-8630-4350-bdcc-f1b390bd6ab9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:54.211463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] The token cache is being invalidate because project 76e1252062164975b8ac89f6181c4c4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:33:54.212199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:54.217634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-08e05b95-8630-4350-bdcc-f1b390bd6ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:33:54.226480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1217/2422] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:33:53 2022] DELETE /identity/v3/projects/76e1252062164975b8ac89f6181c4c4d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:33:54.233789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.234622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.235089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.292491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:54.294690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:54.490949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1206/2423] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:54.498554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.499625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.500525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.549503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:54.551181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:54.567453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1218/2424] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:54.569040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:54.600935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc483f-3daf-4a7f-9f49-63ac31bad9b3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:54.610858 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:54.611567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.612437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.612873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.613864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:33:54.614343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:54.616169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:54.635225 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1207/2425] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:33:54.636897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:54.655711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030fd7-cd6e-478f-abcb-e7393c665c65 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:54.662200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:54.663626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.664190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.664721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.666045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:54.669902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:54.725847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1219/2426] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:33:54 2022] POST /identity/v3/projects => generated 399 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:33:54.732336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:54.744437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd350be-1f41-4d4e-97a2-1888b3ad1931 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:54.861260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:54.862199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.862676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.863168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.864432 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:54.868364 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:54.913851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1208/2427] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:33:54 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:33:54.915889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:54.929157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c697bb-4f92-4f18-8372-f45040fa59c6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:54.934078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:54.934695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.934977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.935240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.935894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:54.937605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:54.952402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1220/2428] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:33:54.953976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:54.970988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f97bc-699b-4ae8-bb36-b4f2f7fe8a69 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:54.979345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c10bbf-7999-40dc-9f4a-db520856c886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:54.979897 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10bbf-7999-40dc-9f4a-db520856c886 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:54.980169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10bbf-7999-40dc-9f4a-db520856c886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:54.980439 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10bbf-7999-40dc-9f4a-db520856c886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:54.981137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c10bbf-7999-40dc-9f4a-db520856c886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:54.982822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c10bbf-7999-40dc-9f4a-db520856c886 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:54.998081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1209/2429] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:33:55.005118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:55.015876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1feb9-22aa-45a3-a575-117797113ffb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:55.022510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:55.023185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:55.023492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:55.023766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] PATH_INFO: `/v3/projects/21b279a7c73b4357bb7ec82e7d609d10/users/da52da164ce34a67a3bf66b54f847e42/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:55.041871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b279a7c73b4357bb7ec82e7d609d10, user_id=da52da164ce34a67a3bf66b54f847e42, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:55.044693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:55.060080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1221/2430] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:33:55 2022] PUT /identity/v3/projects/21b279a7c73b4357bb7ec82e7d609d10/users/da52da164ce34a67a3bf66b54f847e42/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:33:55.071472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8a3d-f1fb-45f7-9314-5ec277138181 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:55.072077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8a3d-f1fb-45f7-9314-5ec277138181 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:55.072608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8a3d-f1fb-45f7-9314-5ec277138181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:55.107331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:33:55.111422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fa4f8a3d-f1fb-45f7-9314-5ec277138181 None None] MFA Rules not processed for user `da52da164ce34a67a3bf66b54f847e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:55.113595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fa4f8a3d-f1fb-45f7-9314-5ec277138181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:55.214295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1210/2431] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:33:55.230554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:33:55.232858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:55.353814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8bd3a-8d54-4d41-8e69-c260f5e59ab3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:33:55.359892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:33:55.361150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:55.361479 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:55.361998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:55.376545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:33:55.378635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:33:55.394975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1222/2432] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:33:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:33:56.532839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:33:56.533737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:33:56.534193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:33:56.554970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] MFA Rules not processed for user `7c5ccd8c99934f60b4ca26c4990b7412`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:33:56.556221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:33:56.614925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1211/2433] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:34:00.304287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:00.305816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:00.330170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:34:00.373256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dd2c7-5601-41b6-b98f-3696bf51bf3c service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:00.378351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:00.379039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:00.379388 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:00.379721 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:00.382158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:00.474017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:00.475632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:00.495567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1223/2434] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:00.896929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:00.898694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:00.912857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179ac6a-cef3-4030-a670-c9c897951b40 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:00.916815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:00.917466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:00.917745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:00.918019 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:00.919812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:00.934683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:00.936145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0242498-55b4-42bd-8486-8855917e4d67 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:00.948529 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1212/2435] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:05.754618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:05.906415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d2c1f-d32a-42df-9417-3e098f6243a8 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:05.915962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:05.917574 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:05.918671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:05.919434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] PATH_INFO: `/v3/users/da52da164ce34a67a3bf66b54f847e42` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:05.922202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da52da164ce34a67a3bf66b54f847e42)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:05.926460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:06.020883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da52da164ce34a67a3bf66b54f847e42'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.032305 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da52da164ce34a67a3bf66b54f847e42'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.033731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da52da164ce34a67a3bf66b54f847e42'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.034542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da52da164ce34a67a3bf66b54f847e42'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.045566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:34:06.049613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da52da164ce34a67a3bf66b54f847e42'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.088707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1224/2436] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:34:05 2022] DELETE /identity/v3/users/da52da164ce34a67a3bf66b54f847e42 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:34:06.094075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:06.094624 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:06.094880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:06.123730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:34:06.125367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:06.221836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1213/2437] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:34:06.235252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:06.238441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:06.373387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fbc39-03d3-4776-af81-b3a270ce1737 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:06.379495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:06.380392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:06.382086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:06.382655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:06.398641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:06.401073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:06.424228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1225/2438] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:06.787137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:06.788831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:06.811975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6322a8f-17eb-484d-b246-52af64b51650 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:06.818846 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:06.819826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:06.820179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:06.820649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] PATH_INFO: `/v3/projects/21b279a7c73b4357bb7ec82e7d609d10` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:06.832943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b279a7c73b4357bb7ec82e7d609d10)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:06.835117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:06.884268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:34:06.909549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b279a7c73b4357bb7ec82e7d609d10', 'request_id': 'req-65871402-a02a-4971-bfe1-f0c1283cb864'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.916926 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b279a7c73b4357bb7ec82e7d609d10', 'request_id': 'req-65871402-a02a-4971-bfe1-f0c1283cb864'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.927044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b279a7c73b4357bb7ec82e7d609d10', 'request_id': 'req-65871402-a02a-4971-bfe1-f0c1283cb864'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.939941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] The token cache is being invalidate because project 21b279a7c73b4357bb7ec82e7d609d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:34:06.940269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.940911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:06.942661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1214/2439] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:34:06 2022] DELETE /identity/v3/projects/21b279a7c73b4357bb7ec82e7d609d10 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:34:15.167016 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:15.168627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:15.183551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa45f5-a978-4e7d-bf3a-b69f42ca8e81 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:15.251588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:15.252149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:15.253107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:15.253412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] PATH_INFO: `/v3/users/d655fb9352544adf8b3bac0cfa202d9e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:15.254636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d655fb9352544adf8b3bac0cfa202d9e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:15.256599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:15.312249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd655fb9352544adf8b3bac0cfa202d9e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.321853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd655fb9352544adf8b3bac0cfa202d9e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.322655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd655fb9352544adf8b3bac0cfa202d9e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.323429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd655fb9352544adf8b3bac0cfa202d9e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.334342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd655fb9352544adf8b3bac0cfa202d9e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.359493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:34:15.365965 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1226/2440] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:34:15 2022] DELETE /identity/v3/users/d655fb9352544adf8b3bac0cfa202d9e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:34:15.584610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:15.586650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:15.605022 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65871402-a02a-4971-bfe1-f0c1283cb864 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:15.681933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:15.682626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:15.682900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:15.683165 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] PATH_INFO: `/v3/projects/64af2cf0e0a14b3ea92c2438baeeb900` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:15.684509 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=64af2cf0e0a14b3ea92c2438baeeb900)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:15.686459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:15.736275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64af2cf0e0a14b3ea92c2438baeeb900', 'request_id': 'req-779f80c7-db17-426c-a944-db6592c111fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.744001 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64af2cf0e0a14b3ea92c2438baeeb900', 'request_id': 'req-779f80c7-db17-426c-a944-db6592c111fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.753368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64af2cf0e0a14b3ea92c2438baeeb900', 'request_id': 'req-779f80c7-db17-426c-a944-db6592c111fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.765358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] The token cache is being invalidate because project 64af2cf0e0a14b3ea92c2438baeeb900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:34:15.765571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.766215 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:34:15.767798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1215/2441] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:34:15 2022] DELETE /identity/v3/projects/64af2cf0e0a14b3ea92c2438baeeb900 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:34:25.761980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:34:43.332762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:43.334586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:43.397147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b75be6-cb84-4753-a770-f893918e40e8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:43.450818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:43.451422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:43.451691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:43.451949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:43.453875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:43.463533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:43.464973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:43.474456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1227/2442] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:45.217942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:45.219985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:45.276220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f80c7-db17-426c-a944-db6592c111fe admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:45.279900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:45.280521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:45.280796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:45.281187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:45.283383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:45.354337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:45.355821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:45.365567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1216/2443] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:46.540972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:46.542164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:46.658864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387c9cc-0c46-4a35-9d08-ee2b0793222d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:46.669486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:46.670387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:46.670745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:46.671096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:46.673781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:46.742253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:34:46.746717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:46.748703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:46.761797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1228/2444] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:47.960543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:47.962044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:47.975749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc555c-c610-408a-aba7-82730bbd4d35 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:47.979449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:47.980050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:47.980327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:47.980599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:47.982399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:47.992205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:47.993804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:48.003031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1217/2445] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:48.314033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:48.316132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:48.415242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1f6ec-ede9-41fb-92f8-e65fd81fde93 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:48.420205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:48.420804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:48.421104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:48.421398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:48.423910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:48.435785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:48.438015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a039235-97bc-4315-95a4-fb89f0fbdfde service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:48.448330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1229/2446] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:52.178023 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:34:52.211010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81800-618d-4078-9271-30aa6970c391 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:34:52.227328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:34:52.228017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:34:52.228394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:34:52.228676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:34:52.231078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:34:52.248513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:34:52.251300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:34:52.268674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1218/2447] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:34:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:34:58.443990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=34 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:00.624983 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:00.625906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:00.625906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:00.646078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] MFA Rules not processed for user `aacb777ebdf445eca64d1e46c2bdeced`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:00.647329 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:00.705962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1230/2448] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:35:01.024706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:01.026184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:01.038010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16d42f-413d-47a9-af2d-aafd095be6d0 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:01.041731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:01.042400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:01.042743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:01.043083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:01.044877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:01.055476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:01.057017 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39718742-d3c3-43f4-8bad-807c76476cf0 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:01.068546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1219/2449] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:35:04.755424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:04.771326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98e65a-38b0-4a13-ba7d-77d4a3b94fad None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:04.777693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:04.778448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:04.781575 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:04.782103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:04.784634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:04.799839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:04.802353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:04.816738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1231/2450] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:35:06.565938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:06.566825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:06.567257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:06.575598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=67 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:06.594405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:06.595682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:06.656062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1220/2451] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:35:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:35:06.710036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:06.711616 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:06.785789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc324ad6-8732-420d-a756-00eb630744b9 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:06.791748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:06.792355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:06.792626 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:06.792892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:06.802161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:06.803931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:06.815430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1232/2452] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:35:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:35:10.955722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:10.982776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab3e70-ca1b-4ad5-ab6a-dba549345132 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:10.991720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:10.992370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:10.992941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:10.993269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:10.995796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:11.129465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:11.131327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:11.145589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1221/2453] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:35:16.086732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:16.088862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:16.108462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8cf64-81f1-4625-86d3-155f27c0167d service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:16.114361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:16.115010 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:16.115283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:16.115550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] PATH_INFO: `/v3/users/7c5ccd8c99934f60b4ca26c4990b7412` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:16.116724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5ccd8c99934f60b4ca26c4990b7412)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:16.118701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:16.184641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5ccd8c99934f60b4ca26c4990b7412'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.204600 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5ccd8c99934f60b4ca26c4990b7412'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.205420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5ccd8c99934f60b4ca26c4990b7412'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.206113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5ccd8c99934f60b4ca26c4990b7412'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.223449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1730509-d5f7-4c6d-b23e-12fe950d2176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5ccd8c99934f60b4ca26c4990b7412'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.236647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:16.252387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1233/2454] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:35:16 2022] DELETE /identity/v3/users/7c5ccd8c99934f60b4ca26c4990b7412 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:35:16.587092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:16.635314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dce9c-9fb7-4e63-8e44-18b6fa1b079f service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:16.751924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:16.755030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:16.755319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:16.755882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] PATH_INFO: `/v3/projects/3dfb62f918dd41aea2e6d49286476dd0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:16.757465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfb62f918dd41aea2e6d49286476dd0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:16.759813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:16.823504 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:16.834476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfb62f918dd41aea2e6d49286476dd0', 'request_id': 'req-afa5fa57-a260-4bf9-b761-0c65076ea0f5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.847346 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfb62f918dd41aea2e6d49286476dd0', 'request_id': 'req-afa5fa57-a260-4bf9-b761-0c65076ea0f5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.856738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfb62f918dd41aea2e6d49286476dd0', 'request_id': 'req-afa5fa57-a260-4bf9-b761-0c65076ea0f5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.870697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] The token cache is being invalidate because project 3dfb62f918dd41aea2e6d49286476dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:35:16.871478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.872375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-afa5fa57-a260-4bf9-b761-0c65076ea0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:35:16.874620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1222/2455] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:35:16 2022] DELETE /identity/v3/projects/3dfb62f918dd41aea2e6d49286476dd0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:35:16.881448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:16.882046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:16.882046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:16.910338 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:16.912369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:17.078365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1234/2456] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:35:17.086041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.086459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.086750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.138911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:17.154834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:17.169086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1223/2457] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:35:17.171765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:17.184570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323850f-7965-44b4-b4ad-04b789bc4b8f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:17.195724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:17.196737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.197139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.197645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.198611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:35:17.199129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:17.201604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:17.221383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1235/2458] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:35:17.225917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:17.235883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc12ec-9c47-4f62-af95-641b21308a87 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:17.242826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:17.244007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.244007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.245140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.245510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:17.251362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:17.285032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1224/2459] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:35:17 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:35:17.289290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:17.300933 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ae616-62e5-44ab-80bb-253d61a805c2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:17.383568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:17.383568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.383568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.383568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.384165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:17.386275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:17.428847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1236/2460] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:35:17 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:35:17.433478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:17.450737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2a93-f427-4873-9917-891016d628bf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:17.471869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:17.472511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.472943 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.474444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.476129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:17.478797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:17.501483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1225/2461] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:35:17.503772 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:17.516133 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b8aa-94c4-4ab7-b5b0-df5cddf34327 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:17.526019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651969e-852b-4c66-8c97-880f183dedb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:17.526643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651969e-852b-4c66-8c97-880f183dedb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.527444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651969e-852b-4c66-8c97-880f183dedb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.527742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651969e-852b-4c66-8c97-880f183dedb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.528396 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651969e-852b-4c66-8c97-880f183dedb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:17.530760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651969e-852b-4c66-8c97-880f183dedb2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:17.548255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1237/2462] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:35:17.552113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:17.571095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410722e5-975b-4b7a-9862-3514ca50d77e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:17.577032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:17.578050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.578610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.579075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] PATH_INFO: `/v3/projects/3a32a1e0f4ae442398d6e179846e1e1b/users/a73941feee334ac294ac875001b148fb/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.604806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a32a1e0f4ae442398d6e179846e1e1b, user_id=a73941feee334ac294ac875001b148fb, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:17.608089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:17.630124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1226/2463] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:35:17 2022] PUT /identity/v3/projects/3a32a1e0f4ae442398d6e179846e1e1b/users/a73941feee334ac294ac875001b148fb/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:35:17.636227 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90844d92-c636-4a35-986f-f0b164f13f4a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:17.636523 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90844d92-c636-4a35-986f-f0b164f13f4a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:17.636819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90844d92-c636-4a35-986f-f0b164f13f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:17.676306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-90844d92-c636-4a35-986f-f0b164f13f4a None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:17.678593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-90844d92-c636-4a35-986f-f0b164f13f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:18.037074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:18.054299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1238/2464] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:35:18.064870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:18.066707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:18.140747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be355cab-07dc-4831-874c-87c654137aff admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:18.145222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:18.145854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:18.146131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:18.146404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:18.158831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:18.160945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:18.173365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1227/2465] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:35:23.639321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55d73-69a6-4454-8508-8b25bad580b6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:23.640160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55d73-69a6-4454-8508-8b25bad580b6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:23.640333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55d73-69a6-4454-8508-8b25bad580b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:23.696788 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-27f55d73-69a6-4454-8508-8b25bad580b6 None None] MFA Rules not processed for user `a73941feee334ac294ac875001b148fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:23.706889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-27f55d73-69a6-4454-8508-8b25bad580b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:23.806784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1239/2466] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:35:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:35:23.822157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:23.824436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:23.932667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964cca8-2cf2-4038-a7dc-efe26172df95 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:23.939614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f08b8-a5c6-467b-9f47-a61857deb73d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:23.940236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f08b8-a5c6-467b-9f47-a61857deb73d service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:23.940510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f08b8-a5c6-467b-9f47-a61857deb73d service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:23.940773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f08b8-a5c6-467b-9f47-a61857deb73d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:23.947520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:23.954929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f08b8-a5c6-467b-9f47-a61857deb73d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:23.967837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f08b8-a5c6-467b-9f47-a61857deb73d service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:23.988806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1228/2467] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:35:26.152671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:26.153240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:26.153240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:26.174216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] MFA Rules not processed for user `a73941feee334ac294ac875001b148fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:26.175319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:26.182351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1240/2468] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:35:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:35:27.449474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74da7d-99e9-413d-b773-de00f7441a8e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:27.450092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74da7d-99e9-413d-b773-de00f7441a8e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:27.451745 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74da7d-99e9-413d-b773-de00f7441a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:27.489331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3f74da7d-99e9-413d-b773-de00f7441a8e None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:27.491915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3f74da7d-99e9-413d-b773-de00f7441a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:27.608642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1229/2469] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:35:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:35:27.625712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:27.627993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:27.715248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c5e47-b4fc-4768-bf1f-f0a65118a73a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:27.722483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:27.723309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:27.723867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:27.724150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:27.741752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:27.744912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:27.759638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1241/2470] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:35:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:35:37.599744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=27 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:37.753528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=67 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:35:44.732666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:44.733145 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:44.733237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:44.752519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:35:44.753750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:44.760746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1230/2471] 10.222.0.85 () {58 vars in 967 bytes} [Wed Nov 2 12:35:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:35:44.773494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:44.784552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf78f2-b40b-4767-b784-6fa0a2984832 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:44.789564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:44.790145 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:44.790418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:44.790685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:44.798825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:44.800435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:44.808639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1242/2472] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:35:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:35:48.996408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:48.997765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:49.014122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456749d4-bd6a-408f-a044-5865abd8ff62 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:49.019771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:49.020368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:49.020638 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:49.020902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:49.022765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:49.036021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:49.037774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:49.048475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1231/2473] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:35:54.329262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:35:54.342267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe897b-f334-45e3-bb3f-df06d42f82cd service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:35:54.348238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:35:54.348825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:35:54.349138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:35:54.349434 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:35:54.351335 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:35:54.412039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:35:54.413690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:35:54.422967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1243/2474] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:35:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:35:59.043782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=9 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:04.418696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=34 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:25.492395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:25.502726 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f03815-13b0-4606-884c-8d9c1804ebcb service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:25.506383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:25.506952 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:25.507224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:25.507491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:25.509228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:25.520969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:25.522400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:25.532069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1232/2475] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:25.652765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:25.654522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:25.666127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1119b-b8dd-49d4-bc56-e8747dca9553 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:25.670555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:25.671150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:25.671508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:25.671671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:25.673381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:25.687100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:25.688495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:25.698175 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1244/2476] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:29.249683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:29.271126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:29.272127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1ef66-756f-496f-86c9-d501a291825c service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:29.272537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:29.277849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:29.278707 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:29.279725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:29.280276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:29.283102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:29.299909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6780a-4bc5-43aa-8348-cabc79a7649c service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:29.330625 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:29.331562 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:29.332044 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:29.332481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] PATH_INFO: `/v3/users/aacb777ebdf445eca64d1e46c2bdeced` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:29.339000 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacb777ebdf445eca64d1e46c2bdeced)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:29.342849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:29.427515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:29.427817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:29.433925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacb777ebdf445eca64d1e46c2bdeced'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.442166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1233/2477] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:29.443518 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:29.445184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:29.458799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacb777ebdf445eca64d1e46c2bdeced'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.459481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacb777ebdf445eca64d1e46c2bdeced'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.460791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacb777ebdf445eca64d1e46c2bdeced'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.487377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacb777ebdf445eca64d1e46c2bdeced'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.545466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1245/2478] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:29 2022] DELETE /identity/v3/users/aacb777ebdf445eca64d1e46c2bdeced => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:36:29.547474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:29.549531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:29.587583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748f5b-de88-48be-88bf-5fb31edd325a service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:29.696099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd323a-5954-4d4c-b1b5-3f322b83e6c0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:29.710861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:29.711628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:29.711962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:29.712302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] PATH_INFO: `/v3/users/83593f8338c444b7b117eeaf3e4cdcd5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:29.715447 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83593f8338c444b7b117eeaf3e4cdcd5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:29.719181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:29.757995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:29.758614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:29.758889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:29.759316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:29.761183 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:29.780764 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:29.784027 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:29.797782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1234/2479] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:29.837679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83593f8338c444b7b117eeaf3e4cdcd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.848856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:29.851205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:29.865220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83593f8338c444b7b117eeaf3e4cdcd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.865898 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83593f8338c444b7b117eeaf3e4cdcd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.869444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83593f8338c444b7b117eeaf3e4cdcd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.874961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff667ea4-e78a-4084-98bf-7c98985ad264 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:29.880031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:29.880784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:29.881150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:29.881470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] PATH_INFO: `/v3/projects/5bc73b74407f4d6184cbd1330ee89145` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:29.882708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc73b74407f4d6184cbd1330ee89145)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:29.884875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:29.885687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:29.889947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12058fa5-af55-4f3b-85a2-514694d826a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83593f8338c444b7b117eeaf3e4cdcd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.901896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:29.919868 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1246/2480] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:29 2022] DELETE /identity/v3/users/83593f8338c444b7b117eeaf3e4cdcd5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:29.948349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc73b74407f4d6184cbd1330ee89145', 'request_id': 'req-e862c60b-153b-4b95-ade1-40242c49248a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.958217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc73b74407f4d6184cbd1330ee89145', 'request_id': 'req-e862c60b-153b-4b95-ade1-40242c49248a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.968591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc73b74407f4d6184cbd1330ee89145', 'request_id': 'req-e862c60b-153b-4b95-ade1-40242c49248a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.985375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] The token cache is being invalidate because project 5bc73b74407f4d6184cbd1330ee89145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:29.985675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.986527 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e862c60b-153b-4b95-ade1-40242c49248a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:29.988241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1235/2481] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:29 2022] DELETE /identity/v3/projects/5bc73b74407f4d6184cbd1330ee89145 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:29.993731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:29.994212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:29.994660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.022313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:30.024526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.112940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1247/2482] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:30.120942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.120942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.121326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.146235 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:30.147676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.157248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.159218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.161569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1236/2483] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:30.165993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.172712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9d4ce-e079-4170-a15d-36269f607451 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.177974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.179398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.179687 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.179963 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] PATH_INFO: `/v3/projects/62aa5934d5d14fdb81dd5d4817ab8e74` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.181405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=62aa5934d5d14fdb81dd5d4817ab8e74)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.184041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.190651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f04093-4342-4819-82e4-9053a675b6eb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.203271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.204680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.205813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.206829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.208257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:36:30.208738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.212884 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.233398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1237/2484] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:36:30.237939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.251207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62aa5934d5d14fdb81dd5d4817ab8e74', 'request_id': 'req-e8217169-27af-46da-9ba8-e4fa4050dd3f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:30.251345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90154ad8-19d1-4c69-adcc-dec5aa1baad6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.263279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62aa5934d5d14fdb81dd5d4817ab8e74', 'request_id': 'req-e8217169-27af-46da-9ba8-e4fa4050dd3f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:30.273991 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62aa5934d5d14fdb81dd5d4817ab8e74', 'request_id': 'req-e8217169-27af-46da-9ba8-e4fa4050dd3f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:30.289976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] The token cache is being invalidate because project 62aa5934d5d14fdb81dd5d4817ab8e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:30.290267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:30.291612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8217169-27af-46da-9ba8-e4fa4050dd3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:30.298029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1248/2485] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:30 2022] DELETE /identity/v3/projects/62aa5934d5d14fdb81dd5d4817ab8e74 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:36:30.299841 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809392-160f-49c9-9d3e-e702e2b2dd63 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.300333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809392-160f-49c9-9d3e-e702e2b2dd63 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.301138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809392-160f-49c9-9d3e-e702e2b2dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.336787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.337723 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.338191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.338621 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.340231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.342504 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5b809392-160f-49c9-9d3e-e702e2b2dd63 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:30.342982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.346964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5b809392-160f-49c9-9d3e-e702e2b2dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.363365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1249/2486] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:30.373424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.373424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.373865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.385607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1238/2487] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/projects => generated 401 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:36:30.389536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.391744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.394871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:30.396069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.454975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:30.473442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1250/2488] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:30.474524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.479487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb9fd1-8d0d-4ac1-95ad-b3322b402987 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.484026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b318e4-87ed-41f2-a6ab-ebe0e66a45f7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.484574 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.485576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.485932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.486275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.486973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.488960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.489444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.490124 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.490474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.490817 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.491546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:36:30.491924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.493620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.502137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:30.512908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1251/2489] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:36:30.515653 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.519313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1239/2490] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/users => generated 301 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:30.523032 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.526870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa83bc-0676-40e2-8965-7ab1510be807 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.532395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.533029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.533318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.533587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.534225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.535412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8c416-e097-449d-9903-6676779dfec8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.536554 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.540694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.541427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.541773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.542111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.542918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.544590 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.558290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1240/2491] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:36:30.561037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.575082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1252/2492] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:36:30.579433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.586205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d635-97c1-48d8-9238-f6b0d33b590a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.656646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8383fe-1eac-4d48-a2e3-b75ef694c963 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.661866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.662484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.662759 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.663026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.663660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.664918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.665642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.665755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.666108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.666476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.667272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.668955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.684787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1241/2493] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:36:30.685962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.690978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1253/2494] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:36:30.692039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.697148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fda6b-69d6-4346-a292-1360cf9d3877 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.702857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241df757-70d3-4770-ae0d-dd729ad4fa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.703796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df757-70d3-4770-ae0d-dd729ad4fa7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.704079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85dfb2-bd66-4b0b-83ed-f9f41f900295 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.704249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df757-70d3-4770-ae0d-dd729ad4fa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.704772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df757-70d3-4770-ae0d-dd729ad4fa7b admin admin] PATH_INFO: `/v3/projects/32c9c5b401004fa08db406e1ad95493d/users/8731020b988343f48071b12a2e4f695a/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.715495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.716392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.716685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.716959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.717792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.720246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.723701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241df757-70d3-4770-ae0d-dd729ad4fa7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c9c5b401004fa08db406e1ad95493d, user_id=8731020b988343f48071b12a2e4f695a, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.727031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241df757-70d3-4770-ae0d-dd729ad4fa7b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.734775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1254/2495] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:36:30.736333 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.748404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc26338-db70-4dec-bf8a-47de66bc02f3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.751513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1242/2496] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:30 2022] PUT /identity/v3/projects/32c9c5b401004fa08db406e1ad95493d/users/8731020b988343f48071b12a2e4f695a/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:30.756827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.756827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.757107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.789450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:30.791109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.818699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.819602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.819881 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.820155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.820976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.822970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.836140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1255/2497] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:36:30.845620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.863444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b782e-b645-435d-b1d1-c4658305cf9a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:30.869508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1243/2498] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:30.875803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a860e47-6b93-4855-ac67-0a84f2972ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:30.877301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a860e47-6b93-4855-ac67-0a84f2972ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.878215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a860e47-6b93-4855-ac67-0a84f2972ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.878835 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a860e47-6b93-4855-ac67-0a84f2972ff2 admin admin] PATH_INFO: `/v3/projects/f7b784528c29493e84202abde804e3b6/users/464a0ab5979549a892aee25bf3004f59/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.885950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:30.892060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.899756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a860e47-6b93-4855-ac67-0a84f2972ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b784528c29493e84202abde804e3b6, user_id=464a0ab5979549a892aee25bf3004f59, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:30.904429 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a860e47-6b93-4855-ac67-0a84f2972ff2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:30.937925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1256/2499] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:30 2022] PUT /identity/v3/projects/f7b784528c29493e84202abde804e3b6/users/464a0ab5979549a892aee25bf3004f59/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:30.942023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:30.942023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:30.942023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:30.970713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:30.983676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] MFA Rules not processed for user `464a0ab5979549a892aee25bf3004f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:30.985474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:30.986466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a21f2-c43f-4222-b6ca-5c89b327a3f1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.012471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:31.123290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.123645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.123924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.124195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.127415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1257/2500] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:31.128566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.222951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49553389-faf9-4087-9e73-509428bfc6bb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.228622 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.230787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.235975 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.237585 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.238957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.239384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.240287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.246690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1244/2501] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:31.250546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.295009 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1258/2502] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:31 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:31.298996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.311839 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004732a-32a5-4e5a-a83b-8682e909b68e service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.395934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.397273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.397273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.397515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.398330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.400761 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.426719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1245/2503] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:31 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:31.431634 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.445028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e0f0c-9a52-42b0-8a24-9f80210be378 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.453312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.454036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.454463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.455446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.456590 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.459855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.486172 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1259/2504] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:31.487526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.502293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bd981-1416-4580-bcec-a0974f7b2550 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.509316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.510275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.510558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.510937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] PATH_INFO: `/v3/projects/43252dc183784758a407ea586f757621/users/a8b880dd56b34e2dab2fd04860bc696e/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.533292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43252dc183784758a407ea586f757621, user_id=a8b880dd56b34e2dab2fd04860bc696e, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.536229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.553944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1246/2505] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:31 2022] PUT /identity/v3/projects/43252dc183784758a407ea586f757621/users/a8b880dd56b34e2dab2fd04860bc696e/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:31.557920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.570390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33febac-af05-44e9-be41-4754a0f69135 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.646720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeec95-0049-41cd-ab3a-60b8772c71b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.658913 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeec95-0049-41cd-ab3a-60b8772c71b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.659263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeec95-0049-41cd-ab3a-60b8772c71b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.659545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeec95-0049-41cd-ab3a-60b8772c71b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.661039 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeec95-0049-41cd-ab3a-60b8772c71b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.662888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeec95-0049-41cd-ab3a-60b8772c71b2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.681323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1260/2506] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:31.682508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.693595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614201ea-27fa-4edd-b5fa-3d74ad2ede45 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.698563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.699532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.699532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.699918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] PATH_INFO: `/v3/projects/43252dc183784758a407ea586f757621/users/a8b880dd56b34e2dab2fd04860bc696e/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.702457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43252dc183784758a407ea586f757621, user_id=a8b880dd56b34e2dab2fd04860bc696e, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:31.706392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:31.719760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1247/2507] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:31 2022] PUT /identity/v3/projects/43252dc183784758a407ea586f757621/users/a8b880dd56b34e2dab2fd04860bc696e/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:31.733057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16166c0-9b9d-4e24-ae23-fa195cced87a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.733548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16166c0-9b9d-4e24-ae23-fa195cced87a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.734246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16166c0-9b9d-4e24-ae23-fa195cced87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.768291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:31.781649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b16166c0-9b9d-4e24-ae23-fa195cced87a None None] MFA Rules not processed for user `a8b880dd56b34e2dab2fd04860bc696e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:31.782895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b16166c0-9b9d-4e24-ae23-fa195cced87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:31.867224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1261/2508] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:31.875271 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:31.877003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:31.899313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:31.958935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f2b7-b40c-4408-b383-36055e588e25 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:31.971920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:31.973073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.973418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.973934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:31.992465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:31.992813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:31.993047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.035385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] MFA Rules not processed for user `8731020b988343f48071b12a2e4f695a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:32.035385 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:32.069430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.071063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.083403 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1248/2509] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:32.121269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1262/2510] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:36:32.125721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:32.225807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a8f9b-9a11-440e-9b85-574527543353 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:32.231570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:32.232501 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:32.232850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:32.233261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.234092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.236247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.284132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1249/2511] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:32 2022] POST /identity/v3/projects => generated 405 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:32.287851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:32.299251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c1d3-5e6f-4fb2-a526-5f7044fd83ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:32.385909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:32.386239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:32.387118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:32.387456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.388181 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.390729 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.416720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1263/2512] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:32 2022] POST /identity/v3/users => generated 309 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:32.420818 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:32.437039 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303453ae-2290-4390-8a2e-d0e4227af70c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:32.443202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:32.443822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:32.444078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:32.444375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.445139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.446696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.459424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1250/2513] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:32.463662 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:32.488694 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4473e7-adf0-4c04-877b-e205f2fde48f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:32.496770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:32.498088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:32.498541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:32.498966 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] PATH_INFO: `/v3/projects/1d9cf4cabae148d3b055f5e02b27fe32/users/c153e3ab165346a2819ac7d2519d4bfe/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.520968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:32.529096 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9cf4cabae148d3b055f5e02b27fe32, user_id=c153e3ab165346a2819ac7d2519d4bfe, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.532645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.555012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1264/2514] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:32 2022] PUT /identity/v3/projects/1d9cf4cabae148d3b055f5e02b27fe32/users/c153e3ab165346a2819ac7d2519d4bfe/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:32.560440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:32.577286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b61ec4-44d5-47e2-9c8e-7cc3948b4e89 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:32.634921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:32.652555 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc66adb-c840-40e1-beab-5523623119a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:32.653204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc66adb-c840-40e1-beab-5523623119a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:32.653485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc66adb-c840-40e1-beab-5523623119a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:32.653757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc66adb-c840-40e1-beab-5523623119a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.654440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc66adb-c840-40e1-beab-5523623119a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.656044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc66adb-c840-40e1-beab-5523623119a3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.666109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1251/2515] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:32.670324 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:32.680849 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5925-3ee3-48da-b802-55f63644f9af admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:32.686334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:32.687099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:32.687375 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:32.687644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] PATH_INFO: `/v3/projects/1d9cf4cabae148d3b055f5e02b27fe32/users/c153e3ab165346a2819ac7d2519d4bfe/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:32.689583 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9cf4cabae148d3b055f5e02b27fe32, user_id=c153e3ab165346a2819ac7d2519d4bfe, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:32.692229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:32.703710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1265/2516] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:32 2022] PUT /identity/v3/projects/1d9cf4cabae148d3b055f5e02b27fe32/users/c153e3ab165346a2819ac7d2519d4bfe/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:33.634937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633cdf0-3a38-4317-96b0-f7188f887b8e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:33.635396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633cdf0-3a38-4317-96b0-f7188f887b8e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:33.635493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633cdf0-3a38-4317-96b0-f7188f887b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:33.668681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0633cdf0-3a38-4317-96b0-f7188f887b8e None None] MFA Rules not processed for user `c153e3ab165346a2819ac7d2519d4bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:33.669986 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0633cdf0-3a38-4317-96b0-f7188f887b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:33.815018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1252/2517] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:35.159617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:35.203543 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:35.204058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:35.204058 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:35.223380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] MFA Rules not processed for user `a73941feee334ac294ac875001b148fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:35.224591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:35.235352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4fbcd-211d-402f-a640-3741213c7bc5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:35.240149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:35.240801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:35.241108 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:35.241376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] PATH_INFO: `/v3/users/8731020b988343f48071b12a2e4f695a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:35.242501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8731020b988343f48071b12a2e4f695a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:35.244411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:35.282305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1253/2518] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:36:35.296642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8731020b988343f48071b12a2e4f695a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.304488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8731020b988343f48071b12a2e4f695a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.305168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8731020b988343f48071b12a2e4f695a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.305797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8731020b988343f48071b12a2e4f695a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.314465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8731020b988343f48071b12a2e4f695a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.332603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1266/2519] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:35 2022] DELETE /identity/v3/users/8731020b988343f48071b12a2e4f695a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:35.523209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:35.535190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf1984-c6cc-40eb-af01-fdcb0122ac31 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:35.606654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:35.607309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:35.607589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:35.607888 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] PATH_INFO: `/v3/projects/32c9c5b401004fa08db406e1ad95493d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:35.609389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c9c5b401004fa08db406e1ad95493d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:35.611315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:35.624710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:35.661944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c9c5b401004fa08db406e1ad95493d', 'request_id': 'req-e731a367-0426-43d7-a93a-884b11306233'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.669123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c9c5b401004fa08db406e1ad95493d', 'request_id': 'req-e731a367-0426-43d7-a93a-884b11306233'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.678028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c9c5b401004fa08db406e1ad95493d', 'request_id': 'req-e731a367-0426-43d7-a93a-884b11306233'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.689731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] The token cache is being invalidate because project 32c9c5b401004fa08db406e1ad95493d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:35.690015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.690672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.692197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1254/2520] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:35 2022] DELETE /identity/v3/projects/32c9c5b401004fa08db406e1ad95493d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:35.696214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:35.697920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:35.711679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7c697-bcb1-4383-9f8a-aa9b2f62fc5a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:35.771532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:35.778961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:35.779914 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:35.780377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:35.780690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] PATH_INFO: `/v3/users/c153e3ab165346a2819ac7d2519d4bfe` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:35.782028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c153e3ab165346a2819ac7d2519d4bfe)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:35.784336 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:35.837031 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c153e3ab165346a2819ac7d2519d4bfe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.845312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c153e3ab165346a2819ac7d2519d4bfe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.846022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c153e3ab165346a2819ac7d2519d4bfe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.846675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c153e3ab165346a2819ac7d2519d4bfe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.854565 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fa24c86b-cead-4a5f-82ab-d26158b45159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c153e3ab165346a2819ac7d2519d4bfe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:35.873376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1267/2521] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:35 2022] DELETE /identity/v3/users/c153e3ab165346a2819ac7d2519d4bfe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:36.058578 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.060164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:36.085962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a367-0426-43d7-a93a-884b11306233 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.158513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.159613 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.160158 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.160692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] PATH_INFO: `/v3/projects/1d9cf4cabae148d3b055f5e02b27fe32` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.162914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9cf4cabae148d3b055f5e02b27fe32)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.166221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.260087 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9cf4cabae148d3b055f5e02b27fe32', 'request_id': 'req-435f88ef-d69b-4bca-9a5e-c5306caa5066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:36.269815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9cf4cabae148d3b055f5e02b27fe32', 'request_id': 'req-435f88ef-d69b-4bca-9a5e-c5306caa5066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:36.279570 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9cf4cabae148d3b055f5e02b27fe32', 'request_id': 'req-435f88ef-d69b-4bca-9a5e-c5306caa5066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:36.305702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] The token cache is being invalidate because project 1d9cf4cabae148d3b055f5e02b27fe32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:36.306024 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:36.306828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-435f88ef-d69b-4bca-9a5e-c5306caa5066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:36.308666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1255/2522] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:36 2022] DELETE /identity/v3/projects/1d9cf4cabae148d3b055f5e02b27fe32 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:36.314576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.315081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.315246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.336874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:36.338462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:36.402846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1268/2523] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:36.410373 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.410654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.410929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.435177 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:36.436334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:36.457101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1256/2524] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:36.461762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.472582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a132-cfa7-43ab-b71b-8a7eacf610ce None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.477341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.477951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.478232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.478506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.479154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:36:36.479453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.481690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.493764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1269/2525] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:36:36.498391 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.507749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9719e75-1b0c-466b-ad0a-75b0b4efaa2e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.512662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.513371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.513710 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.514044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.514744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.516796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.545149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1257/2526] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:36 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:36.549502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.560103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce349b9-4e50-46fc-8e23-4c5dd065d3cf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.624205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.624862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.625215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.625486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.626125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.628023 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.638750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:36.654288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1270/2527] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:36 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:36.655522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.662660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:36.669274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216f67e-a7cb-4caf-9b36-424512cf3748 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.674797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.675415 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.676246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.676246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.678300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.678392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.690124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1258/2528] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:36.694314 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.706541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291c64-08f0-4a3f-b25d-a33881db4f03 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.711461 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3e0e9-6257-4777-86d9-3e27f652536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.712059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3e0e9-6257-4777-86d9-3e27f652536a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.712341 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3e0e9-6257-4777-86d9-3e27f652536a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.712611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3e0e9-6257-4777-86d9-3e27f652536a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.713291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3e0e9-6257-4777-86d9-3e27f652536a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.715576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3e0e9-6257-4777-86d9-3e27f652536a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.727072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1271/2529] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:36.731363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:36.741917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b49ea2-5107-4410-a2ee-91c97f42b605 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:36.747295 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:36.748304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.748584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.748859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] PATH_INFO: `/v3/projects/6acd45604d2f4249aad24f0e14d78614/users/18182dcdd6314b0e934588a1fe624466/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.765198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6acd45604d2f4249aad24f0e14d78614, user_id=18182dcdd6314b0e934588a1fe624466, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:36.767854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:36.781608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1259/2530] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:36 2022] PUT /identity/v3/projects/6acd45604d2f4249aad24f0e14d78614/users/18182dcdd6314b0e934588a1fe624466/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:36.792154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:36.792364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:36.792619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:36.821950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] MFA Rules not processed for user `18182dcdd6314b0e934588a1fe624466`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:36.823205 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:36.921816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1272/2531] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:36.926747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.004416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc262e-6169-4d4f-89d6-f249a58592cc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.008640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.010690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:37.012089 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.012989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.013368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.013689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.014397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.016690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.059834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1260/2532] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:36 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:37.063997 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.091775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247b395-63bf-4f20-b382-823b395b3369 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.131677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5367e6-7402-4247-a50e-1d95da93ddd3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.136484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.137122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.137400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.137668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.138299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.140211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.143712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.144310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.144579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.144842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.146638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:37.167925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1261/2533] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:37 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:36:37.168984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.179093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f808b6-c41e-4027-8fb0-42f17881616e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.184337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.184933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.185245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.185535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.186192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.187808 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.193293 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:37.200040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1262/2534] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:37.204304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.214811 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3fd26-b3b7-405a-9018-b1508796506c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.218149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.219802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.220121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.220319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.220626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.220967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] PATH_INFO: `/v3/projects/e19410bb60a04838af49f9bae3cc3ed7/users/b198c7ee3d404465a4a25f265f7d884f/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.230508 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1273/2535] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:36:37.237645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19410bb60a04838af49f9bae3cc3ed7, user_id=b198c7ee3d404465a4a25f265f7d884f, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.240350 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.268097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1263/2536] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:37 2022] PUT /identity/v3/projects/e19410bb60a04838af49f9bae3cc3ed7/users/b198c7ee3d404465a4a25f265f7d884f/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:37.272138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.289075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ed0a1-0ba0-41d2-8756-30f4b726797c service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.382777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567505-fd01-4ceb-9a66-68ef4d2bb7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.384577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567505-fd01-4ceb-9a66-68ef4d2bb7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.385407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567505-fd01-4ceb-9a66-68ef4d2bb7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.387979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567505-fd01-4ceb-9a66-68ef4d2bb7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.388819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567505-fd01-4ceb-9a66-68ef4d2bb7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.391496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567505-fd01-4ceb-9a66-68ef4d2bb7a3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.415492 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1274/2537] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:37.415905 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.432428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665888f-632a-4198-8676-9ce418330796 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.449417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.452924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.453310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.454522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] PATH_INFO: `/v3/projects/e19410bb60a04838af49f9bae3cc3ed7/users/b198c7ee3d404465a4a25f265f7d884f/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.459720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19410bb60a04838af49f9bae3cc3ed7, user_id=b198c7ee3d404465a4a25f265f7d884f, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.466915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.471963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:37.504921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1264/2538] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:37 2022] PUT /identity/v3/projects/e19410bb60a04838af49f9bae3cc3ed7/users/b198c7ee3d404465a4a25f265f7d884f/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:36:37.529246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.529246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.529246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.588062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] MFA Rules not processed for user `b198c7ee3d404465a4a25f265f7d884f`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:37.589568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:37.681948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1275/2539] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:37.693958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:37.695639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:37.764524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d6050-5559-4d10-93cf-e669f0450baa admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:37.768115 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7985b-efde-401d-b301-db4b36e4d5fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:37.768704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7985b-efde-401d-b301-db4b36e4d5fb service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:37.768976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7985b-efde-401d-b301-db4b36e4d5fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:37.769262 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7985b-efde-401d-b301-db4b36e4d5fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:37.778716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7985b-efde-401d-b301-db4b36e4d5fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:37.780140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7985b-efde-401d-b301-db4b36e4d5fb service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:37.790198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1265/2540] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:36:38.398650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:38.484277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742a86e-3e76-4ad8-9e7b-23bc8f44c8ee None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:38.489534 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:38.490245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:38.490528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:38.490801 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] PATH_INFO: `/v3/users/18182dcdd6314b0e934588a1fe624466` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:38.491993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18182dcdd6314b0e934588a1fe624466)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:38.494229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:38.496851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:38.556309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18182dcdd6314b0e934588a1fe624466'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:38.568292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18182dcdd6314b0e934588a1fe624466'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:38.568292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18182dcdd6314b0e934588a1fe624466'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:38.568292 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18182dcdd6314b0e934588a1fe624466'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:38.578754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18182dcdd6314b0e934588a1fe624466'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:38.603796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1276/2541] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:38 2022] DELETE /identity/v3/users/18182dcdd6314b0e934588a1fe624466 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:38.608566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:38.609535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:38.609967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:38.634857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:38.636356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:38.712106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1266/2542] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:38.724470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:38.726331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:38.844848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab981b60-92ec-4425-8b9f-5ae2963a84d9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:38.850714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:38.851403 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:38.851889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:38.852262 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:38.859247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:38.863911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:38.865377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:38.869442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076c7a6-581c-4309-ad5c-1a027d894514 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:38.874009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:38.874608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:38.874881 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:38.875451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:38.876666 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1277/2543] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:38.877756 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:38.899050 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:38.900496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:38.910694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1267/2544] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:36:39.294525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:39.296029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:39.310572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9062-cc61-4152-a7f1-593c10651ee1 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:39.316483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:39.317194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:39.317470 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:39.317737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] PATH_INFO: `/v3/projects/6acd45604d2f4249aad24f0e14d78614` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:39.318889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6acd45604d2f4249aad24f0e14d78614)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:39.321482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:39.374883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6acd45604d2f4249aad24f0e14d78614', 'request_id': 'req-21b12026-8af3-49ae-a6a7-cd653a49a40f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.381246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6acd45604d2f4249aad24f0e14d78614', 'request_id': 'req-21b12026-8af3-49ae-a6a7-cd653a49a40f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.389164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6acd45604d2f4249aad24f0e14d78614', 'request_id': 'req-21b12026-8af3-49ae-a6a7-cd653a49a40f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.400358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] The token cache is being invalidate because project 6acd45604d2f4249aad24f0e14d78614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:39.400977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.401410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.403715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1278/2545] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:39 2022] DELETE /identity/v3/projects/6acd45604d2f4249aad24f0e14d78614 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:39.407807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:39.418806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874d39c-f57f-40c1-aad7-09a1655e56b2 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:39.475841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:39.476489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:39.476767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:39.477068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] PATH_INFO: `/v3/users/b198c7ee3d404465a4a25f265f7d884f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:39.478256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b198c7ee3d404465a4a25f265f7d884f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:39.480815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:39.511449 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=1,keystone:SELECT=95,keystone:DELETE=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:39.532916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b198c7ee3d404465a4a25f265f7d884f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.540735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b198c7ee3d404465a4a25f265f7d884f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.541383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b198c7ee3d404465a4a25f265f7d884f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.542002 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b198c7ee3d404465a4a25f265f7d884f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.549820 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9203a54c-02cc-4a3c-bafd-18b02fb2b636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b198c7ee3d404465a4a25f265f7d884f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:39.568213 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1268/2546] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:39 2022] DELETE /identity/v3/users/b198c7ee3d404465a4a25f265f7d884f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:39.868935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:39.870871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:39.882395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:39.886808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12026-8af3-49ae-a6a7-cd653a49a40f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:39.975584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:39.976281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:39.976610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:39.976838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] PATH_INFO: `/v3/projects/e19410bb60a04838af49f9bae3cc3ed7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:39.978678 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19410bb60a04838af49f9bae3cc3ed7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:39.980283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.040627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19410bb60a04838af49f9bae3cc3ed7', 'request_id': 'req-0e440e89-4db8-4d28-902b-4e379c90bebe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:40.049024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19410bb60a04838af49f9bae3cc3ed7', 'request_id': 'req-0e440e89-4db8-4d28-902b-4e379c90bebe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:40.060301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19410bb60a04838af49f9bae3cc3ed7', 'request_id': 'req-0e440e89-4db8-4d28-902b-4e379c90bebe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:40.075062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] The token cache is being invalidate because project e19410bb60a04838af49f9bae3cc3ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:40.075517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:40.076395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e440e89-4db8-4d28-902b-4e379c90bebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:40.078163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1279/2547] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:39 2022] DELETE /identity/v3/projects/e19410bb60a04838af49f9bae3cc3ed7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:40.088672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.089821 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.090273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.111224 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:40.112782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:40.190575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1269/2548] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:40.198458 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.199339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.199611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.220159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:40.221454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:40.230007 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1280/2549] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:40.234160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.244341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee445-bc54-48da-929a-5f4cf2c2f2c5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.250098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.250719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.251042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.251333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.252075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:36:40.252398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.254126 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.267853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1270/2550] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:36:40.272367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.282449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ca08a-d492-45aa-9dca-d01dd95b5718 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.287337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.287944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.288251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.288540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.289198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.291116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.320962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1281/2551] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:40 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:40.325267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.336695 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83dddb-c45c-494d-8ef4-d75d88b1dd6c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.394579 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.394931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.395208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.395478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.396114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.398065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.420553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1271/2552] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:40 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:40.425441 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.437506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d0e16-016e-4195-b6a1-238d66575f7d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.442465 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.443162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.443442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.443738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.444411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.446353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.457689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1282/2553] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:40.461673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.474444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1ae08-1e4f-42fe-a7b7-1519d063bd80 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.479283 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748f761-42c8-4d05-9e7a-966e249c23d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.480836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748f761-42c8-4d05-9e7a-966e249c23d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.481056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748f761-42c8-4d05-9e7a-966e249c23d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.481330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748f761-42c8-4d05-9e7a-966e249c23d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.481970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748f761-42c8-4d05-9e7a-966e249c23d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.483571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748f761-42c8-4d05-9e7a-966e249c23d7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.499947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1272/2554] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:40.502146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.514521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692971-ccbe-41ac-936f-988050a28508 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.521120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.521909 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.522302 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.522660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] PATH_INFO: `/v3/projects/f7fb3e7854aa47c9a1b4cb496dc7cd4f/users/783e559b08cb44e3b478fb546bf6f40a/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.543372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7fb3e7854aa47c9a1b4cb496dc7cd4f, user_id=783e559b08cb44e3b478fb546bf6f40a, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.546976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.574705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1283/2555] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:40 2022] PUT /identity/v3/projects/f7fb3e7854aa47c9a1b4cb496dc7cd4f/users/783e559b08cb44e3b478fb546bf6f40a/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:36:40.576219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.576511 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.576786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.608920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:40.628419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] MFA Rules not processed for user `783e559b08cb44e3b478fb546bf6f40a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:40.629737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:40.702409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1273/2556] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:40.715671 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:40.717542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:40.778710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:40.812825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4322757-fe09-481e-9ad0-75a02f5c9103 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:40.818789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:40.819095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:40.819412 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:40.819638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:40.830065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:40.831545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:40.849547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1284/2557] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:46.493475 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:46.494900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:46.562706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2530-5be4-4031-bfdb-9683e879cc6f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:46.567382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:46.568010 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:46.568285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:46.568558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] PATH_INFO: `/v3/users/20a4049d2b2b474a8a830d0702b9b1fa` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:46.569713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a4049d2b2b474a8a830d0702b9b1fa)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:46.571631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:46.622405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a4049d2b2b474a8a830d0702b9b1fa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.631526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a4049d2b2b474a8a830d0702b9b1fa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.632149 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a4049d2b2b474a8a830d0702b9b1fa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.632750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a4049d2b2b474a8a830d0702b9b1fa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.639953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a4049d2b2b474a8a830d0702b9b1fa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.657819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1274/2558] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:46 2022] DELETE /identity/v3/users/20a4049d2b2b474a8a830d0702b9b1fa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:46.836435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:46.848528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bc5c9-b3ec-49fc-9384-a22e599aee75 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:46.906327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:46.906872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:46.907149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:46.907418 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] PATH_INFO: `/v3/projects/620e39db57f34948b5450e5b90c984cb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:46.908536 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=620e39db57f34948b5450e5b90c984cb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:46.910484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:46.958106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620e39db57f34948b5450e5b90c984cb', 'request_id': 'req-e850b4ca-6df3-49ee-8b3b-62070d273f10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.964636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620e39db57f34948b5450e5b90c984cb', 'request_id': 'req-e850b4ca-6df3-49ee-8b3b-62070d273f10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.972100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620e39db57f34948b5450e5b90c984cb', 'request_id': 'req-e850b4ca-6df3-49ee-8b3b-62070d273f10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.983173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] The token cache is being invalidate because project 620e39db57f34948b5450e5b90c984cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:46.983576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.984214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:46.985797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1285/2559] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:46 2022] DELETE /identity/v3/projects/620e39db57f34948b5450e5b90c984cb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:46.989793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:46.991193 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:47.004760 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc97be-e8f2-4d80-837e-e11de0989374 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:47.029255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:47.078389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:47.079026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:47.079305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:47.079576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] PATH_INFO: `/v3/users/1219613bc6c649aca2b85ed5ed05d4fc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:47.080715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1219613bc6c649aca2b85ed5ed05d4fc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:47.082663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:47.130217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1219613bc6c649aca2b85ed5ed05d4fc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.137360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1219613bc6c649aca2b85ed5ed05d4fc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.138040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1219613bc6c649aca2b85ed5ed05d4fc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.138684 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1219613bc6c649aca2b85ed5ed05d4fc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.146751 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1219613bc6c649aca2b85ed5ed05d4fc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.166252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1275/2560] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:46 2022] DELETE /identity/v3/users/1219613bc6c649aca2b85ed5ed05d4fc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:47.338509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:47.340660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:47.354105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850b4ca-6df3-49ee-8b3b-62070d273f10 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:47.417502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:47.418062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:47.418337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:47.418606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] PATH_INFO: `/v3/projects/b7b9eca8e3f046fb82ba57f7aca35983` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:47.419886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b9eca8e3f046fb82ba57f7aca35983)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:47.421842 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:47.428430 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:47.477945 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b9eca8e3f046fb82ba57f7aca35983', 'request_id': 'req-0e79613c-adfa-47fd-a2df-3956b3a35cb2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.484826 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b9eca8e3f046fb82ba57f7aca35983', 'request_id': 'req-0e79613c-adfa-47fd-a2df-3956b3a35cb2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.492087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b9eca8e3f046fb82ba57f7aca35983', 'request_id': 'req-0e79613c-adfa-47fd-a2df-3956b3a35cb2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.502642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] The token cache is being invalidate because project b7b9eca8e3f046fb82ba57f7aca35983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:47.502921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.503538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e79613c-adfa-47fd-a2df-3956b3a35cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:47.505017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1286/2561] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:47 2022] DELETE /identity/v3/projects/b7b9eca8e3f046fb82ba57f7aca35983 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:53.785250 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:53.786772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:53.867899 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e9fe-4a56-478f-82e3-0d93aaef2fc0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:53.900563 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:53.927002 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:53.927639 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:53.927918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:53.928192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] PATH_INFO: `/v3/users/783e559b08cb44e3b478fb546bf6f40a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:53.929402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=783e559b08cb44e3b478fb546bf6f40a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:53.931441 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:53.979152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783e559b08cb44e3b478fb546bf6f40a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:53.986536 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783e559b08cb44e3b478fb546bf6f40a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:53.987147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783e559b08cb44e3b478fb546bf6f40a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:53.987824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783e559b08cb44e3b478fb546bf6f40a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:53.994958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783e559b08cb44e3b478fb546bf6f40a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:54.012887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1276/2562] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:36:53 2022] DELETE /identity/v3/users/783e559b08cb44e3b478fb546bf6f40a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:54.017727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.018209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.018209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.046502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:54.047710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:54.105890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1287/2563] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:54.115690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:54.117113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:54.191358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd921-353c-4238-b3e7-2943256618d8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:54.195055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2a2e7-d79a-4d61-b04b-0979946e46c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:54.195657 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2a2e7-d79a-4d61-b04b-0979946e46c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.195932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2a2e7-d79a-4d61-b04b-0979946e46c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.196200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2a2e7-d79a-4d61-b04b-0979946e46c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.205790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2a2e7-d79a-4d61-b04b-0979946e46c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:54.207218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2a2e7-d79a-4d61-b04b-0979946e46c9 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:54.217409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1277/2564] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:36:54.576138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:54.582197 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:54.594731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115faae8-0215-4a18-a833-d608ef3bda75 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:54.602046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:54.602791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.603111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.603481 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] PATH_INFO: `/v3/projects/f7fb3e7854aa47c9a1b4cb496dc7cd4f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.604798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7fb3e7854aa47c9a1b4cb496dc7cd4f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:54.607893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:54.662313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7fb3e7854aa47c9a1b4cb496dc7cd4f', 'request_id': 'req-51083580-9d59-491c-baf1-c6c2c7a23d22'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:54.669582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7fb3e7854aa47c9a1b4cb496dc7cd4f', 'request_id': 'req-51083580-9d59-491c-baf1-c6c2c7a23d22'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:54.680072 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7fb3e7854aa47c9a1b4cb496dc7cd4f', 'request_id': 'req-51083580-9d59-491c-baf1-c6c2c7a23d22'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:54.692748 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] The token cache is being invalidate because project f7fb3e7854aa47c9a1b4cb496dc7cd4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:36:54.693137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:54.693844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-51083580-9d59-491c-baf1-c6c2c7a23d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:36:54.695563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1288/2565] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:36:54 2022] DELETE /identity/v3/projects/f7fb3e7854aa47c9a1b4cb496dc7cd4f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:54.701725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.701980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.702257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.760759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:54.762198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:54.775774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:54.830838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1278/2566] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:54.839572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.839952 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.840372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.860838 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:54.862100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:54.870456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1289/2567] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:54.875259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:54.885517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a42b5f-58a2-4b35-ae39-1fb3f90b3225 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:54.891593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:54.892111 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.892387 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.892658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.893354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:36:54.893661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:54.895277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:54.907545 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1279/2568] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:36:54.912164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:54.919141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:54.923456 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916eaec1-13f5-4977-ba20-92bef1bfe1b7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:54.929381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:54.930160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:54.930650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:54.930987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:54.931811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:54.934045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:54.961992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1290/2569] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:54 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:36:54.966894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:54.977217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c4e60-0c79-414f-80ff-c127acc42d63 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.034345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.034951 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.035227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.035496 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.036124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.038110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.059353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1280/2570] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:54 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:55.063740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.074603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ca971-20eb-4585-9f2b-6925909e7305 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.079179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.079763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.080035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.080301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.081046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.082552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.093128 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1291/2571] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:55.097491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.107461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8ae52-3e05-41c0-8cca-cdcc57a4f29a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.112218 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b8ffd-a82b-462c-809b-b83343f63cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.112803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b8ffd-a82b-462c-809b-b83343f63cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.113133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b8ffd-a82b-462c-809b-b83343f63cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.113409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b8ffd-a82b-462c-809b-b83343f63cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.114074 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b8ffd-a82b-462c-809b-b83343f63cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.115672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b8ffd-a82b-462c-809b-b83343f63cc5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.126961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1281/2572] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:55.140462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.151981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51782a87-1a35-4f97-8f77-666f7b1c8e83 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.156744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.157414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.157690 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.157960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] PATH_INFO: `/v3/projects/599f1da8b1434287ae992c4194179a84/users/8a2ec687355e483b9def3a3fcf60472b/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.172585 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=599f1da8b1434287ae992c4194179a84, user_id=8a2ec687355e483b9def3a3fcf60472b, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.175251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.188146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1292/2573] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:55 2022] PUT /identity/v3/projects/599f1da8b1434287ae992c4194179a84/users/8a2ec687355e483b9def3a3fcf60472b/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:55.199152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.199434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.199708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.227804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] MFA Rules not processed for user `8a2ec687355e483b9def3a3fcf60472b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:55.229065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:55.293854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1282/2574] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:55.298339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.361551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67871de9-5583-4c2d-aa58-e32ac49a90fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.366101 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.366702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.366970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.367235 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.367874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.370415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.397281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1293/2575] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:55 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:55.401532 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.412315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc8b0a-c56c-40ef-b4d4-b97536d22fa2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.421399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:55.474376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.474993 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.475270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.475542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.476190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.478160 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.506963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1283/2576] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:55 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:55.508263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.521494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efddbaf-8ee0-4167-91ca-18c3fcbb6c8a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.527118 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.527978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.528378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.528777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.529538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.531231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.544696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1294/2577] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:36:55.549652 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.559845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4ed27-5065-4b75-8bf4-44c085e578c0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.566485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0751f-025e-4c6c-bfe9-6586ede108e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.567454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0751f-025e-4c6c-bfe9-6586ede108e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.568064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0751f-025e-4c6c-bfe9-6586ede108e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.568342 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0751f-025e-4c6c-bfe9-6586ede108e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.569025 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0751f-025e-4c6c-bfe9-6586ede108e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.570677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0751f-025e-4c6c-bfe9-6586ede108e1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.589026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1284/2578] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:55.593183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.607639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fc248-5cb6-4685-9f5d-da3c45fc5186 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.613934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.614807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.615089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.615368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] PATH_INFO: `/v3/projects/046b1e694c0144348123a7108d2933c4/users/728813403f2442ac864d364fa27b7791/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.633533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046b1e694c0144348123a7108d2933c4, user_id=728813403f2442ac864d364fa27b7791, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.637036 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.664524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1295/2579] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:55 2022] PUT /identity/v3/projects/046b1e694c0144348123a7108d2933c4/users/728813403f2442ac864d364fa27b7791/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:36:55.670587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.670889 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.671672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.708720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] MFA Rules not processed for user `728813403f2442ac864d364fa27b7791`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:55.710035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:55.801971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1285/2580] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:36:55.806641 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.875961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:55.879862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c27665-74eb-4123-9f4f-c01134289d28 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.886742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.887394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.888564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.888843 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.889876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.891805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:55.921029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1296/2581] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:36:55 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:55.925413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:55.936877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725647-4b80-4513-a03d-dfcc5ebdfa7f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:55.992870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:55.993495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:55.993773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:55.994035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:55.994741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:55.996670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:56.018495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1286/2582] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:36:55 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:36:56.022500 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:56.033503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412e265-e05d-4d4d-9625-7a72ad1167d9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:56.038173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:56.038756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:56.039030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:56.039294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:56.039921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:56.041545 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:56.052437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1297/2583] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:56.056537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:56.067191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0eac1-1af9-4e0c-9d45-1caa6810660c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:56.071843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:56.072480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:56.073414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:56.073716 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] PATH_INFO: `/v3/projects/851a7e83f95e4fdbaa4aa127c1cfe27e/users/efdeddb3408542109cc5235b5b7e68b1/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:56.082144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:36:56.089379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=851a7e83f95e4fdbaa4aa127c1cfe27e, user_id=efdeddb3408542109cc5235b5b7e68b1, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:56.092201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:56.104679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1287/2584] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:56 2022] PUT /identity/v3/projects/851a7e83f95e4fdbaa4aa127c1cfe27e/users/efdeddb3408542109cc5235b5b7e68b1/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:56.108645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:56.118731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beeeb64-b17f-411c-99e4-5d6fa1ddb5ac admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:56.174732 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd0c97-2cea-43a6-9272-458cad25d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:56.175287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0c97-2cea-43a6-9272-458cad25d5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:56.175611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0c97-2cea-43a6-9272-458cad25d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:56.175931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0c97-2cea-43a6-9272-458cad25d5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:56.176572 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd0c97-2cea-43a6-9272-458cad25d5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:56.178213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd0c97-2cea-43a6-9272-458cad25d5a4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:56.188639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1298/2585] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:36:56.192909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:56.204253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61df3a-c862-46fe-b508-f0fe54c01d66 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:56.208927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:56.209628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:56.209904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:56.210175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] PATH_INFO: `/v3/projects/851a7e83f95e4fdbaa4aa127c1cfe27e/users/efdeddb3408542109cc5235b5b7e68b1/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:56.212083 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=851a7e83f95e4fdbaa4aa127c1cfe27e, user_id=efdeddb3408542109cc5235b5b7e68b1, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:56.214787 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:56.224937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1288/2586] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:36:56 2022] PUT /identity/v3/projects/851a7e83f95e4fdbaa4aa127c1cfe27e/users/efdeddb3408542109cc5235b5b7e68b1/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:36:56.235300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:56.235752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:56.236161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:56.266114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] MFA Rules not processed for user `efdeddb3408542109cc5235b5b7e68b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:36:56.267331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:56.334760 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1299/2587] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:36:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:36:56.346686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:36:56.348419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:36:56.406797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04896f-690f-4668-a2f7-b3c242c8dbed admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:36:56.410644 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:36:56.411332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:36:56.411637 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:36:56.411851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:36:56.470995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:36:56.472505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:36:56.482724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1289/2588] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:36:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:00.142306 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:00.144200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:00.161029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abb2e7-0f0a-4f35-b34f-57385e9f76e6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:00.165249 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:00.166218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:00.167482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:00.167845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:00.220285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:00.230584 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:00.232798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:00.242906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1300/2589] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:00.888284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:00.898680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57638d91-6a3c-4ad7-8eb3-60a77d17e29b service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:00.902901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af799c42-a1cb-43a7-9278-cac4c3528c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:00.903661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af799c42-a1cb-43a7-9278-cac4c3528c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:00.904033 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af799c42-a1cb-43a7-9278-cac4c3528c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:00.904537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af799c42-a1cb-43a7-9278-cac4c3528c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:00.915450 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af799c42-a1cb-43a7-9278-cac4c3528c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:00.917663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af799c42-a1cb-43a7-9278-cac4c3528c8c service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:00.931237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1290/2590] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:03.886486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:03.888541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:03.964093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6737e-2364-4109-8ab9-400b504bef7e service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:03.971469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:03.972526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:03.973060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:03.973547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] PATH_INFO: `/v3/users/464a0ab5979549a892aee25bf3004f59` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:03.975438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=464a0ab5979549a892aee25bf3004f59)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:03.979643 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:04.032696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464a0ab5979549a892aee25bf3004f59'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.041106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464a0ab5979549a892aee25bf3004f59'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.041867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464a0ab5979549a892aee25bf3004f59'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.042445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464a0ab5979549a892aee25bf3004f59'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.049996 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464a0ab5979549a892aee25bf3004f59'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.067710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1301/2591] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:03 2022] DELETE /identity/v3/users/464a0ab5979549a892aee25bf3004f59 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:04.072875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:04.073425 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:04.073549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:04.093452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:04.094660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:04.153219 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1291/2592] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:04.163170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:04.164695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:04.222752 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee307f29-29b3-405e-ac87-e3c13a81de28 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:04.226496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:04.227086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:04.227357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:04.227773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:04.236699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:04.238129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:04.247629 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1302/2593] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:04.517479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:04.518909 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:04.531461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bf05f-d096-4db5-b1ae-f4cda0e6b04b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:04.536505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:04.537358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:04.537634 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:04.537904 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] PATH_INFO: `/v3/projects/f7b784528c29493e84202abde804e3b6` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:04.539030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b784528c29493e84202abde804e3b6)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:04.541044 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:04.553749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:04.592495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b784528c29493e84202abde804e3b6', 'request_id': 'req-4460bf8e-8fb8-4aa8-b366-14ddd93af612'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.599336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b784528c29493e84202abde804e3b6', 'request_id': 'req-4460bf8e-8fb8-4aa8-b366-14ddd93af612'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.606841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b784528c29493e84202abde804e3b6', 'request_id': 'req-4460bf8e-8fb8-4aa8-b366-14ddd93af612'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.617344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] The token cache is being invalidate because project f7b784528c29493e84202abde804e3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:04.617627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.618249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.620239 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1292/2594] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:04 2022] DELETE /identity/v3/projects/f7b784528c29493e84202abde804e3b6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:04.624459 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:04.633950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea4dd4-c7f7-43f0-8fbb-dca44bd87576 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:04.643682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:04.695855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:04.696482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:04.696758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:04.697131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] PATH_INFO: `/v3/users/a8b880dd56b34e2dab2fd04860bc696e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:04.698256 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b880dd56b34e2dab2fd04860bc696e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:04.700163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:04.748408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b880dd56b34e2dab2fd04860bc696e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.755812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b880dd56b34e2dab2fd04860bc696e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.756578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b880dd56b34e2dab2fd04860bc696e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.757255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b880dd56b34e2dab2fd04860bc696e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.764853 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2eca2972-148a-41e7-a859-d534438ed988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b880dd56b34e2dab2fd04860bc696e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.783289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1303/2595] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:04 2022] DELETE /identity/v3/users/a8b880dd56b34e2dab2fd04860bc696e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:04.833606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:04.835278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:04.850101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460bf8e-8fb8-4aa8-b366-14ddd93af612 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:04.917069 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:04.918114 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:04.918484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:04.918980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] PATH_INFO: `/v3/users/8a2ec687355e483b9def3a3fcf60472b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:04.920308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a2ec687355e483b9def3a3fcf60472b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:04.922696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:04.987516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a2ec687355e483b9def3a3fcf60472b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.998428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a2ec687355e483b9def3a3fcf60472b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.999161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a2ec687355e483b9def3a3fcf60472b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:04.999835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a2ec687355e483b9def3a3fcf60472b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.007400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a2ec687355e483b9def3a3fcf60472b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.026876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1293/2596] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:04 2022] DELETE /identity/v3/users/8a2ec687355e483b9def3a3fcf60472b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:05.031537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.032045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.032140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.051485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:05.052715 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.081791 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.085108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.113525 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:05.130934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1304/2597] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:05.142241 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.144258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.174269 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844d6e0-d70b-4030-9d47-9bed29fa8bb9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.179837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.180485 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.180757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.181046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] PATH_INFO: `/v3/projects/43252dc183784758a407ea586f757621` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.182730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43252dc183784758a407ea586f757621)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.185054 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.185211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:05.213346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a516d9d-1831-479f-9db6-0d24affb2b01 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.219599 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23fb33-3366-4c37-8f45-d58ee3129a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.221438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23fb33-3366-4c37-8f45-d58ee3129a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.221961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23fb33-3366-4c37-8f45-d58ee3129a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.222440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23fb33-3366-4c37-8f45-d58ee3129a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.233114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23fb33-3366-4c37-8f45-d58ee3129a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.234555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23fb33-3366-4c37-8f45-d58ee3129a3b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.236429 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43252dc183784758a407ea586f757621', 'request_id': 'req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.243978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43252dc183784758a407ea586f757621', 'request_id': 'req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.245213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1305/2598] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:37:05.254005 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43252dc183784758a407ea586f757621', 'request_id': 'req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.267517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] The token cache is being invalidate because project 43252dc183784758a407ea586f757621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:05.267833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.268572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-711f5a24-3ac9-4a15-9a32-0fe81fcb8f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.275865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1294/2599] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:05 2022] DELETE /identity/v3/projects/43252dc183784758a407ea586f757621 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:05.281217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.281501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.281773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.304608 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:05.306168 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.373889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1306/2600] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:05.381969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.382251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.382523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.403110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:05.404345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.416815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1295/2601] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:05.421808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.432576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce7fe3-3539-4387-926f-58d478e7f26b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.437669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.438372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.438711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.439047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.439795 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:05.440157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.442491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.454829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1307/2602] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:37:05.459173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.468933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2ea9d-f0c1-4eb1-870f-f74ba3c1fd3c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.473667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.474352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.474627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.474894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.475542 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.477455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.509034 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1296/2603] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:05.512173 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.523272 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4c1ae-39ed-47cf-bd0b-6fa6271cffdf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.548383 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.550357 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.597930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.598537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.598816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.599089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.599722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.602298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.624904 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1308/2604] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:05.626498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac81cd-920b-4416-b9c2-6b601f5131f6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.628738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.631436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.632075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.632355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.632630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] PATH_INFO: `/v3/projects/599f1da8b1434287ae992c4194179a84` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.633859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=599f1da8b1434287ae992c4194179a84)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.636318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.639524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab68d63-2b95-4c00-aeb3-6e1f3f81255d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.644693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.645323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.645601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.645896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.646533 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.648116 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.660879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1309/2605] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:37:05.661611 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:05.665274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.676056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1a92e-02f3-46fe-acbc-6209cfa64fe1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.687914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599f1da8b1434287ae992c4194179a84', 'request_id': 'req-e2e2b832-3508-44ce-aeee-4e7c86e37559'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.694377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599f1da8b1434287ae992c4194179a84', 'request_id': 'req-e2e2b832-3508-44ce-aeee-4e7c86e37559'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.702880 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599f1da8b1434287ae992c4194179a84', 'request_id': 'req-e2e2b832-3508-44ce-aeee-4e7c86e37559'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.715140 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] The token cache is being invalidate because project 599f1da8b1434287ae992c4194179a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:05.715430 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.716088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.717945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1297/2606] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:05 2022] DELETE /identity/v3/projects/599f1da8b1434287ae992c4194179a84 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:05.720611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:05.721944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.723540 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.744765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.745471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.745754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.746114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.746758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.748765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.759771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1310/2607] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:05.763871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.765736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.778111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad91a79-d778-41b3-be87-f5f24130079b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.782768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356ef12-0121-4690-877a-1bd148653434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.783651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356ef12-0121-4690-877a-1bd148653434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.783651 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356ef12-0121-4690-877a-1bd148653434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.783940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356ef12-0121-4690-877a-1bd148653434 admin admin] PATH_INFO: `/v3/projects/a4d2bdac6d274d928397d12e3d0aac24/users/b3d4bf7a6f6a4800afed5aebccaa2731/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.796486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2b832-3508-44ce-aeee-4e7c86e37559 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:05.799515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356ef12-0121-4690-877a-1bd148653434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d2bdac6d274d928397d12e3d0aac24, user_id=b3d4bf7a6f6a4800afed5aebccaa2731, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.801998 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:05.802179 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356ef12-0121-4690-877a-1bd148653434 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.802630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.802929 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.803186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] PATH_INFO: `/v3/users/728813403f2442ac864d364fa27b7791` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.804341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=728813403f2442ac864d364fa27b7791)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:05.806289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:05.815519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1311/2608] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:05 2022] PUT /identity/v3/projects/a4d2bdac6d274d928397d12e3d0aac24/users/b3d4bf7a6f6a4800afed5aebccaa2731/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:05.827621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:05.827901 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:05.828174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:05.862198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] MFA Rules not processed for user `b3d4bf7a6f6a4800afed5aebccaa2731`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:05.862414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728813403f2442ac864d364fa27b7791'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.863436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.870206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728813403f2442ac864d364fa27b7791'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.870958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728813403f2442ac864d364fa27b7791'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.871645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728813403f2442ac864d364fa27b7791'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.879974 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728813403f2442ac864d364fa27b7791'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:05.899155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1298/2609] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:05 2022] DELETE /identity/v3/users/728813403f2442ac864d364fa27b7791 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:05.935896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1312/2610] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:05.937021 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:05.938508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:05.973691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:06.014221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfba330-ed0c-4551-b783-2b2777dbf4d3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.018965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.019567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.019843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.020108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.020755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.022729 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.057894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1299/2611] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:05 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:06.059093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.135860 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b0ec-3490-4881-a229-21373a7ca1f0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.196195 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:06.210068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.211827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:06.214864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.215573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.215902 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.216239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.216948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.219079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.237505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdc5fe-e3c0-4ae5-b987-18bf677d7f07 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.245418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.246422 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.246779 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.247577 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] PATH_INFO: `/v3/projects/046b1e694c0144348123a7108d2933c4` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.247931 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1313/2612] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:06 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:06.252029 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.252200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=046b1e694c0144348123a7108d2933c4)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.254313 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.264744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7d49-4777-415c-96b0-a2d4a546bea5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.273825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.273825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.273825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.273825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.274271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.274988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.286874 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1314/2613] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:37:06.291155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.302789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5dfa9-5b64-4dee-b3eb-19644a0f5b89 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.309845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.310520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.310827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.311125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] PATH_INFO: `/v3/projects/36ca5719322c4f9a84214d9e1d4cb490/users/802aa0bfe36b4e899bdd486eaf7fa7ea/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.319212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046b1e694c0144348123a7108d2933c4', 'request_id': 'req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.327533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046b1e694c0144348123a7108d2933c4', 'request_id': 'req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.330915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ca5719322c4f9a84214d9e1d4cb490, user_id=802aa0bfe36b4e899bdd486eaf7fa7ea, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.333774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.337324 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046b1e694c0144348123a7108d2933c4', 'request_id': 'req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.350807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1315/2614] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:06 2022] PUT /identity/v3/projects/36ca5719322c4f9a84214d9e1d4cb490/users/802aa0bfe36b4e899bdd486eaf7fa7ea/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:06.353082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] The token cache is being invalidate because project 046b1e694c0144348123a7108d2933c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:06.353408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.354257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.355773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1300/2615] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:06 2022] DELETE /identity/v3/projects/046b1e694c0144348123a7108d2933c4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:06.357431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.359598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:06.360339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.361715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:06.445100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06f93e-8172-4a25-aca9-9e2cacce1c2d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.446789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5592-1c77-4e42-ae28-05d80d833d8b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.450895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.452167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.452167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.452167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] PATH_INFO: `/v3/users/efdeddb3408542109cc5235b5b7e68b1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.453286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.453286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.453534 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=efdeddb3408542109cc5235b5b7e68b1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.453620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.453620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.454311 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.455322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.455922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.472059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1316/2616] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:06.473454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.486057 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e69910-1c5e-44de-8d31-6d2e99726a1c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.492684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346ef81-a3e9-402f-b673-2116659de849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.493376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ef81-a3e9-402f-b673-2116659de849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.493698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ef81-a3e9-402f-b673-2116659de849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.493936 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ef81-a3e9-402f-b673-2116659de849 admin admin] PATH_INFO: `/v3/projects/36ca5719322c4f9a84214d9e1d4cb490/users/802aa0bfe36b4e899bdd486eaf7fa7ea/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.496219 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346ef81-a3e9-402f-b673-2116659de849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ca5719322c4f9a84214d9e1d4cb490, user_id=802aa0bfe36b4e899bdd486eaf7fa7ea, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.499717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346ef81-a3e9-402f-b673-2116659de849 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.513321 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1317/2617] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:06 2022] PUT /identity/v3/projects/36ca5719322c4f9a84214d9e1d4cb490/users/802aa0bfe36b4e899bdd486eaf7fa7ea/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:06.524123 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.524415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.524646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:06.524789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.535513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efdeddb3408542109cc5235b5b7e68b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.545310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efdeddb3408542109cc5235b5b7e68b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.546267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efdeddb3408542109cc5235b5b7e68b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.547155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efdeddb3408542109cc5235b5b7e68b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.559375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efdeddb3408542109cc5235b5b7e68b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:06.559724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] MFA Rules not processed for user `802aa0bfe36b4e899bdd486eaf7fa7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:06.561040 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:06.589443 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1301/2618] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:06 2022] DELETE /identity/v3/users/efdeddb3408542109cc5235b5b7e68b1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:06.650998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1318/2619] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:06.664586 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.666985 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:06.742305 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271266e5-4ba7-46cc-b1db-b74fd30ad01a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:06.747754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e5ce9-c289-45b3-83d1-fa31dd4014aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:06.748609 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e5ce9-c289-45b3-83d1-fa31dd4014aa service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:06.748878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e5ce9-c289-45b3-83d1-fa31dd4014aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:06.749181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e5ce9-c289-45b3-83d1-fa31dd4014aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:06.812335 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e5ce9-c289-45b3-83d1-fa31dd4014aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:06.813907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e5ce9-c289-45b3-83d1-fa31dd4014aa service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:06.824690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1302/2620] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:06.908797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:06.910911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:06.936327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:06.999977 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6754a83-fd58-43b4-9f8f-3e5db4a55722 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.064140 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.064791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.065068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.065340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] PATH_INFO: `/v3/projects/851a7e83f95e4fdbaa4aa127c1cfe27e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.066497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=851a7e83f95e4fdbaa4aa127c1cfe27e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.068432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.126570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851a7e83f95e4fdbaa4aa127c1cfe27e', 'request_id': 'req-d7168a05-8b1d-4b74-b6ed-9874436bb441'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.133020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851a7e83f95e4fdbaa4aa127c1cfe27e', 'request_id': 'req-d7168a05-8b1d-4b74-b6ed-9874436bb441'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.142737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851a7e83f95e4fdbaa4aa127c1cfe27e', 'request_id': 'req-d7168a05-8b1d-4b74-b6ed-9874436bb441'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.154741 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] The token cache is being invalidate because project 851a7e83f95e4fdbaa4aa127c1cfe27e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:07.155053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.155672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.157290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1319/2621] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:06 2022] DELETE /identity/v3/projects/851a7e83f95e4fdbaa4aa127c1cfe27e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:07.164204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe039e-99dc-48a6-8e10-16746d0b63e4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.164645 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe039e-99dc-48a6-8e10-16746d0b63e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.164836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe039e-99dc-48a6-8e10-16746d0b63e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.191466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8efe039e-99dc-48a6-8e10-16746d0b63e4 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:07.192711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8efe039e-99dc-48a6-8e10-16746d0b63e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:07.206972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.208581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:07.263841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1303/2622] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:07.267161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:07.271914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.272187 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.272461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.288813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:07.293865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7168a05-8b1d-4b74-b6ed-9874436bb441 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.296007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:07.298185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:07.300742 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '802aa0bfe36b4e899bdd486eaf7fa7ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ca5719322c4f9a84214d9e1d4cb490', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.301516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.301820 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.302114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.302761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:07.303068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.304657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.308395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1304/2623] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:07.313428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.318688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1320/2624] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:37:07.322911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.326257 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c4ec-1f53-4d7b-b682-8eeffeb2cc30 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.331564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.332168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.332477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.332750 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.332953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9bc3-c9ff-4a45-addf-23c01a195562 tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.333444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:07.333749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.335360 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.338620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '802aa0bfe36b4e899bdd486eaf7fa7ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ca5719322c4f9a84214d9e1d4cb490', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.339218 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.339496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.339799 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.340426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.342374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.352824 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1305/2625] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:37:07.354636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.366056 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006307-19e4-44da-b223-5beeecac633f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.371247 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.371853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.372389 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.372901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.373046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1321/2626] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:07 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:07.373416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.375514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.376949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.407617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1306/2627] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:07 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:07.412434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.449232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9343f1-aa92-4e92-bd03-0de651a6df6d tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.482777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c56e-6198-4166-890a-9b98fb928b3f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.490477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.491097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.491376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.491678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.492322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.494286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.510373 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '802aa0bfe36b4e899bdd486eaf7fa7ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ca5719322c4f9a84214d9e1d4cb490', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.511045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.511322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.511593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.512547 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:07.512688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.514318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.520547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1307/2628] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:07 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:07.521941 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.527992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1322/2629] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:37:07.533241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc8c1b-088f-4424-8c64-04e50ed63cb0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.538626 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.539459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.539738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.540008 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.540696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.542921 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.554174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1308/2630] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:37:07.560281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.567026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.571537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8d339-db35-4eab-b86e-4ce0ad3e2b4f tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.576586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.577250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.577542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.577811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.578485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.580106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.580807 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a9129-07e2-4e23-b192-af4156921ff2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.586486 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '802aa0bfe36b4e899bdd486eaf7fa7ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ca5719322c4f9a84214d9e1d4cb490', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.587315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.588325 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.588631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] PATH_INFO: `/v3/projects/0e1cc1a24d0542f5a65d2cddb5fb7bdb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.589991 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e1cc1a24d0542f5a65d2cddb5fb7bdb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.591992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.592679 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1323/2631] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:37:07.596927 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.609932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6479eb-2527-452a-b5f2-1d5848f9d21c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.618682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2ad9b-95ad-4689-9720-e18a31255c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.619393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2ad9b-95ad-4689-9720-e18a31255c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.619664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2ad9b-95ad-4689-9720-e18a31255c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.619940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2ad9b-95ad-4689-9720-e18a31255c23 admin admin] PATH_INFO: `/v3/projects/2b3b8f23a66e4540a0f65330c6d6496e/users/ff4810b8d4ef4c80a47b3208ce5439d5/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.639705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2ad9b-95ad-4689-9720-e18a31255c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3b8f23a66e4540a0f65330c6d6496e, user_id=ff4810b8d4ef4c80a47b3208ce5439d5, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.642449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2ad9b-95ad-4689-9720-e18a31255c23 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.656920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1324/2632] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:07 2022] PUT /identity/v3/projects/2b3b8f23a66e4540a0f65330c6d6496e/users/ff4810b8d4ef4c80a47b3208ce5439d5/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:07.662159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.662525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.662873 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.673298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1cc1a24d0542f5a65d2cddb5fb7bdb', 'request_id': 'req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.680772 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1cc1a24d0542f5a65d2cddb5fb7bdb', 'request_id': 'req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.682644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:07.683862 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:07.689343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1cc1a24d0542f5a65d2cddb5fb7bdb', 'request_id': 'req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.702755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] The token cache is being invalidate because project 0e1cc1a24d0542f5a65d2cddb5fb7bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:07.703041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.703754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.705366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1309/2633] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:07 2022] DELETE /identity/v3/projects/0e1cc1a24d0542f5a65d2cddb5fb7bdb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:07.711561 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:07.754056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1325/2634] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:07.764636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.766241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:07.800776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:07.830334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e6fd-4dc0-43aa-95a0-4c8b05bc416c tempest-VolumeQuotaClassesTest-535351711 tempest-VolumeQuotaClassesTest-535351711-project-admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.834537 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc71a-5bc3-4948-be6e-9756cad2263b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.835221 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc71a-5bc3-4948-be6e-9756cad2263b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.835524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc71a-5bc3-4948-be6e-9756cad2263b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.843176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc71a-5bc3-4948-be6e-9756cad2263b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.846402 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bc71a-5bc3-4948-be6e-9756cad2263b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.847845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bc71a-5bc3-4948-be6e-9756cad2263b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.857696 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1310/2635] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:07.918281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:07.920180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:07.932063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fe152-c641-428c-973d-9e36f7dbfab7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:07.937265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:07.937884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:07.938161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:07.938436 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] PATH_INFO: `/v3/users/b3d4bf7a6f6a4800afed5aebccaa2731` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:07.939611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d4bf7a6f6a4800afed5aebccaa2731)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:07.941586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:07.990560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d4bf7a6f6a4800afed5aebccaa2731'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:07.999951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d4bf7a6f6a4800afed5aebccaa2731'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.000677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d4bf7a6f6a4800afed5aebccaa2731'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.001444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d4bf7a6f6a4800afed5aebccaa2731'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.009722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d4bf7a6f6a4800afed5aebccaa2731'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.028086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1326/2636] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:07 2022] DELETE /identity/v3/users/b3d4bf7a6f6a4800afed5aebccaa2731 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:08.032572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:08.032855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:08.033232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:08.055378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:08.057633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:08.121399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1311/2637] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:08.131728 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:08.133332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:08.195646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f8e22-2d2a-4e72-8cdf-6fac239ea48c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:08.199878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:08.200609 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:08.200892 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:08.201297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:08.213840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:08.215861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:08.228407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1327/2638] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:08.734676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:08.736244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:08.750433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f23d6-a021-463a-8a5d-a1cbcb0064e6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:08.757564 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:08.758372 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:08.758653 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:08.758945 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] PATH_INFO: `/v3/projects/a4d2bdac6d274d928397d12e3d0aac24` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:08.760323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d2bdac6d274d928397d12e3d0aac24)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:08.762222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:08.838980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d2bdac6d274d928397d12e3d0aac24', 'request_id': 'req-97817fdb-1f64-4a86-8b8b-c826e9e51520'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.849619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d2bdac6d274d928397d12e3d0aac24', 'request_id': 'req-97817fdb-1f64-4a86-8b8b-c826e9e51520'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.863237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d2bdac6d274d928397d12e3d0aac24', 'request_id': 'req-97817fdb-1f64-4a86-8b8b-c826e9e51520'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.885339 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] The token cache is being invalidate because project a4d2bdac6d274d928397d12e3d0aac24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:08.885831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.887037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:08.889095 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1312/2639] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:08 2022] DELETE /identity/v3/projects/a4d2bdac6d274d928397d12e3d0aac24 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:08.895093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:08.910544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66050417-150a-43da-9d92-21f7b233d901 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:08.974555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:08.999655 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:08.999992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:09.000376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:09.000803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] PATH_INFO: `/v3/users/802aa0bfe36b4e899bdd486eaf7fa7ea` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:09.002203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=802aa0bfe36b4e899bdd486eaf7fa7ea)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:09.005170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:09.096162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802aa0bfe36b4e899bdd486eaf7fa7ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.108130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802aa0bfe36b4e899bdd486eaf7fa7ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.109392 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802aa0bfe36b4e899bdd486eaf7fa7ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.110082 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802aa0bfe36b4e899bdd486eaf7fa7ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.122557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-617902ed-1012-4855-942f-da81238bbc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802aa0bfe36b4e899bdd486eaf7fa7ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.154106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1328/2640] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:08 2022] DELETE /identity/v3/users/802aa0bfe36b4e899bdd486eaf7fa7ea => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:09.535788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:09.537505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:09.565776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97817fdb-1f64-4a86-8b8b-c826e9e51520 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:09.673030 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:09.690963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:09.691718 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:09.692055 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:09.692399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] PATH_INFO: `/v3/projects/36ca5719322c4f9a84214d9e1d4cb490` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:09.693777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ca5719322c4f9a84214d9e1d4cb490)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:09.696699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:09.762632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ca5719322c4f9a84214d9e1d4cb490', 'request_id': 'req-75a29438-6942-43a8-aa16-843351307c07'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.771612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ca5719322c4f9a84214d9e1d4cb490', 'request_id': 'req-75a29438-6942-43a8-aa16-843351307c07'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.783077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ca5719322c4f9a84214d9e1d4cb490', 'request_id': 'req-75a29438-6942-43a8-aa16-843351307c07'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.798490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] The token cache is being invalidate because project 36ca5719322c4f9a84214d9e1d4cb490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:09.798875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.799727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-75a29438-6942-43a8-aa16-843351307c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:09.801782 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1313/2641] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:09 2022] DELETE /identity/v3/projects/36ca5719322c4f9a84214d9e1d4cb490 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:09.808266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:09.808549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:09.808825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:09.832976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:09.834374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:09.968169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1329/2642] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:09.981796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b32275-4414-408b-9066-681862ace0e9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:09.982287 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b32275-4414-408b-9066-681862ace0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:09.982765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b32275-4414-408b-9066-681862ace0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.012776 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-10b32275-4414-408b-9066-681862ace0e9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:10.014121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-10b32275-4414-408b-9066-681862ace0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:10.026873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1314/2643] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:10.027476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.039027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d40b5-8851-4512-b53c-95e8c42ebfc9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.047422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.048199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.048487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.048792 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.052155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:10.052155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.052155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.070393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1330/2644] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:37:10.078007 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b32275-4414-408b-9066-681862ace0e9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.092355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b32275-4414-408b-9066-681862ace0e9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.099129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.100064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.100551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.101080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.102277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.105813 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.140804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1315/2645] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:10 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:10.142034 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.153216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b005bb-a0ce-4207-bb58-85c4cb028640 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.207457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:10.219558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.220656 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.222203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.222480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.223213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.225203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.260953 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1331/2646] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:10 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:10.268828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.280728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055b5e1-1fe2-4fe0-a1a9-11dc470079e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.287192 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.287925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.288206 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.288562 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.289437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.291959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.306670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1316/2647] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:10.307951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.319522 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515cd32-8b14-48c8-8e7b-6e0e5fa99f41 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.324845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbffb16-a998-4a17-9ad3-ea1bba99dc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.325589 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbffb16-a998-4a17-9ad3-ea1bba99dc0c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.325944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbffb16-a998-4a17-9ad3-ea1bba99dc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.326332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbffb16-a998-4a17-9ad3-ea1bba99dc0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.327059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbffb16-a998-4a17-9ad3-ea1bba99dc0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.328829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbffb16-a998-4a17-9ad3-ea1bba99dc0c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.345443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1332/2648] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:10.347755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.361143 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f9d4-5d46-4b5c-b784-eb6cb284e8ba admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.365958 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.366623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.366927 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.367207 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] PATH_INFO: `/v3/projects/a5e70e106c474f70b857f8126a3b8a46/users/35352b78b6ad481f8fe31a5c072edac7/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.395142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e70e106c474f70b857f8126a3b8a46, user_id=35352b78b6ad481f8fe31a5c072edac7, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.400769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.422797 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1317/2649] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:10 2022] PUT /identity/v3/projects/a5e70e106c474f70b857f8126a3b8a46/users/35352b78b6ad481f8fe31a5c072edac7/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:10.436255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.436525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.436809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.480815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] MFA Rules not processed for user `35352b78b6ad481f8fe31a5c072edac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:10.482379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:10.567770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1333/2650] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:10.575488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.667836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e64ef9-f987-4543-8908-c542ad14749d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.674103 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.675493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.675493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.675865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.676665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.679887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.726845 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1318/2651] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:10 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:10.729080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.746253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda972a-4667-49c1-a2aa-1857ad4f262e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.843060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.844284 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.844871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.845527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.846390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.848887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.886979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1334/2652] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:10 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:10.888473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.901770 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:10.908275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1c949-b2cf-4b83-ba0c-392110b2d535 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.921064 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.923048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.923048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.923472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:10.925104 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:10.929855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:10.944291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1319/2653] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:10.949393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:10.967528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8560c3a-9aa8-42d8-a62b-489b771c986a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:10.979504 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:10.980265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:10.980539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:10.980811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] PATH_INFO: `/v3/projects/4f81a6f7acf44b4fa668272127864bf1/users/6fe458f742394dd0bbd839a43ed0ae7b/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:11.012837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f81a6f7acf44b4fa668272127864bf1, user_id=6fe458f742394dd0bbd839a43ed0ae7b, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:11.016222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:11.037075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1335/2654] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:10 2022] PUT /identity/v3/projects/4f81a6f7acf44b4fa668272127864bf1/users/6fe458f742394dd0bbd839a43ed0ae7b/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:11.044168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:11.061713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f326d-4eb0-4b69-9197-f3d2de6fe28a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:11.202006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b445-9a92-44b2-afec-55cc2f4de736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:11.203121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b445-9a92-44b2-afec-55cc2f4de736 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:11.203725 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b445-9a92-44b2-afec-55cc2f4de736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:11.204280 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b445-9a92-44b2-afec-55cc2f4de736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:11.205411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b445-9a92-44b2-afec-55cc2f4de736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:11.207520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b445-9a92-44b2-afec-55cc2f4de736 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:11.224512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1320/2655] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:11.236183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:11.251586 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b9a68-dc5b-4d2b-8941-81bda7a5b235 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:11.259117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:11.261041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:11.261041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:11.261041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] PATH_INFO: `/v3/projects/4f81a6f7acf44b4fa668272127864bf1/users/6fe458f742394dd0bbd839a43ed0ae7b/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:11.265069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f81a6f7acf44b4fa668272127864bf1, user_id=6fe458f742394dd0bbd839a43ed0ae7b, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:11.271254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:11.295233 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1336/2656] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:11 2022] PUT /identity/v3/projects/4f81a6f7acf44b4fa668272127864bf1/users/6fe458f742394dd0bbd839a43ed0ae7b/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:11.310713 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:11.311070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:11.311362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:11.357963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] MFA Rules not processed for user `6fe458f742394dd0bbd839a43ed0ae7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:11.359608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:11.453270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1321/2657] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:11.469147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:11.471204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:11.501830 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:11.590528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194ed15-130b-44d8-aa6e-0bd3bb647d11 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:11.598939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:11.600435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:11.602366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:11.602366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:11.621907 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:11.626015 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:11.649857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1337/2658] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:11.788535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:11.802279 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3963ca-947f-4ec0-9a07-ec34e8763e83 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:11.806873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb8ca5-c095-48a5-b9ee-2b6169bd54c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:11.807604 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb8ca5-c095-48a5-b9ee-2b6169bd54c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:11.807953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb8ca5-c095-48a5-b9ee-2b6169bd54c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:11.808301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb8ca5-c095-48a5-b9ee-2b6169bd54c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:11.914134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb8ca5-c095-48a5-b9ee-2b6169bd54c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:11.917999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb8ca5-c095-48a5-b9ee-2b6169bd54c1 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:11.929046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1322/2659] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:12.306521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:12.453618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326e6e2-42e3-45ec-bc05-70a2a6637e97 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:12.463104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:12.465521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:12.472117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:12.472117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] PATH_INFO: `/v3/users/35352b78b6ad481f8fe31a5c072edac7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:12.475904 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35352b78b6ad481f8fe31a5c072edac7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:12.479519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:12.570453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35352b78b6ad481f8fe31a5c072edac7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:12.588903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35352b78b6ad481f8fe31a5c072edac7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:12.590087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35352b78b6ad481f8fe31a5c072edac7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:12.591097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35352b78b6ad481f8fe31a5c072edac7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:12.601831 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35352b78b6ad481f8fe31a5c072edac7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:12.635094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1338/2660] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:12 2022] DELETE /identity/v3/users/35352b78b6ad481f8fe31a5c072edac7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:12.641478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064eb664-cc3c-4b17-aa22-674ac20a988b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:12.641803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064eb664-cc3c-4b17-aa22-674ac20a988b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:12.642312 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064eb664-cc3c-4b17-aa22-674ac20a988b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:12.673702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-064eb664-cc3c-4b17-aa22-674ac20a988b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:12.676737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-064eb664-cc3c-4b17-aa22-674ac20a988b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:12.697610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:12.793658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1323/2661] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:12.810389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:12.811887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:12.892110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:12.953425 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ec668-c038-433a-b2ca-f5d772edb827 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:12.958130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:12.958922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:12.959207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:12.959619 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:12.972334 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:12.973839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:12.989819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1339/2662] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:13.447117 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.447870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.448094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.490194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] MFA Rules not processed for user `ff4810b8d4ef4c80a47b3208ce5439d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:13.491575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:13.553220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:13.555290 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:13.590973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a323d-88af-45f4-85ca-589473013fa7 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:13.602202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:13.603141 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.603455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.605230 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] PATH_INFO: `/v3/projects/a5e70e106c474f70b857f8126a3b8a46` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.606684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e70e106c474f70b857f8126a3b8a46)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:13.609211 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:13.614267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1324/2663] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:13.626932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:13.630780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:13.647670 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a2b82-0590-4b55-9b85-7000b7883ebe None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:13.656011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:13.656796 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.657131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.657437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.658176 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:13.660127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:13.698446 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1325/2664] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:13 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:13.700505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e70e106c474f70b857f8126a3b8a46', 'request_id': 'req-15885b36-834c-41fa-b710-5472ec216ed1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:13.702680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:13.713895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e70e106c474f70b857f8126a3b8a46', 'request_id': 'req-15885b36-834c-41fa-b710-5472ec216ed1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:13.728630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e70e106c474f70b857f8126a3b8a46', 'request_id': 'req-15885b36-834c-41fa-b710-5472ec216ed1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:13.752668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] The token cache is being invalidate because project a5e70e106c474f70b857f8126a3b8a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:13.753114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:13.754094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:13.756743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1340/2665] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:13 2022] DELETE /identity/v3/projects/a5e70e106c474f70b857f8126a3b8a46 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:13.762597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:13.764323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:13.823355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c270-0503-4e1a-91a1-5f0496efff6e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:13.834936 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:13.835950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.837062 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.837740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.839674 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:13.843201 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:13.875770 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15885b36-834c-41fa-b710-5472ec216ed1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:13.880418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1326/2666] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:13 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:13.886780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:13.889156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:13.889554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:13.890151 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.890673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.891146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] PATH_INFO: `/v3/users/6fe458f742394dd0bbd839a43ed0ae7b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.892825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe458f742394dd0bbd839a43ed0ae7b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:13.895087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:13.904480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45038d8-2a0e-42a0-ba0e-0c3f7332f81a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:13.910301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:13.910933 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.911210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.911487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.912151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:13.913878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:13.925328 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:13.939769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1327/2667] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:37:13.944267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:13.962595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18fd7d-bec2-49b1-82a5-7d6182ccc007 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:13.968731 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:13.969599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:13.969954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:13.970299 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] PATH_INFO: `/v3/projects/daf9d2d6e1ee4ec09ab4a8ef642f93ec/users/c1175476deaf4f14967accc136a0b823/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:13.983444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:13.994804 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe458f742394dd0bbd839a43ed0ae7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.004392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf9d2d6e1ee4ec09ab4a8ef642f93ec, user_id=c1175476deaf4f14967accc136a0b823, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:14.006937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe458f742394dd0bbd839a43ed0ae7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.007818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe458f742394dd0bbd839a43ed0ae7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.008614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe458f742394dd0bbd839a43ed0ae7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.011138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:14.021115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe458f742394dd0bbd839a43ed0ae7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.031968 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1328/2668] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:13 2022] PUT /identity/v3/projects/daf9d2d6e1ee4ec09ab4a8ef642f93ec/users/c1175476deaf4f14967accc136a0b823/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:14.036667 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:14.039134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:14.049834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1341/2669] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:13 2022] DELETE /identity/v3/users/6fe458f742394dd0bbd839a43ed0ae7b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:14.174381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f593d43-eb1a-4207-b492-adec10867f11 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:14.299704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:14.300618 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:14.301454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:14.312681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:14.313473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:14.315166 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:14.344270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1329/2670] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:37:14.348783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:14.369485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04718919-4488-4eb8-bf21-84ad7f63b0c0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:14.376362 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26a0c-57a3-4988-91a5-981312913587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:14.377395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26a0c-57a3-4988-91a5-981312913587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:14.378497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26a0c-57a3-4988-91a5-981312913587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:14.378893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26a0c-57a3-4988-91a5-981312913587 admin admin] PATH_INFO: `/v3/projects/daf9d2d6e1ee4ec09ab4a8ef642f93ec/users/c1175476deaf4f14967accc136a0b823/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:14.381352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26a0c-57a3-4988-91a5-981312913587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf9d2d6e1ee4ec09ab4a8ef642f93ec, user_id=c1175476deaf4f14967accc136a0b823, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:14.384424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26a0c-57a3-4988-91a5-981312913587 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:14.402875 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1342/2671] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:14 2022] PUT /identity/v3/projects/daf9d2d6e1ee4ec09ab4a8ef642f93ec/users/c1175476deaf4f14967accc136a0b823/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:14.587870 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:14.590004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:14.609338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65727-3930-467e-979d-51c715c24b92 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:14.723320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:14.724973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:14.724973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:14.724973 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] PATH_INFO: `/v3/projects/4f81a6f7acf44b4fa668272127864bf1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:14.729159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f81a6f7acf44b4fa668272127864bf1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:14.730148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:14.793900 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:14.824875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f81a6f7acf44b4fa668272127864bf1', 'request_id': 'req-683e5b3b-824c-4c5d-8439-acfb4f131548'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.834544 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f81a6f7acf44b4fa668272127864bf1', 'request_id': 'req-683e5b3b-824c-4c5d-8439-acfb4f131548'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.846970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f81a6f7acf44b4fa668272127864bf1', 'request_id': 'req-683e5b3b-824c-4c5d-8439-acfb4f131548'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.860144 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] The token cache is being invalidate because project 4f81a6f7acf44b4fa668272127864bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:14.860479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.861459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-683e5b3b-824c-4c5d-8439-acfb4f131548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:14.867426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1330/2672] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:14 2022] DELETE /identity/v3/projects/4f81a6f7acf44b4fa668272127864bf1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:14.870915 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:14.871764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:14.872398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:14.899683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:14.901528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:15.010242 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1343/2673] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:15.013314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.014258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.014258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.050957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:15.053123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:15.076479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1331/2674] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:15.083917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.103159 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dd697-c38d-40c3-bd61-c079b564b0d3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.108332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c962cc-478b-447c-9105-826962101899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.108978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c962cc-478b-447c-9105-826962101899 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.109286 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c962cc-478b-447c-9105-826962101899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.109552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c962cc-478b-447c-9105-826962101899 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.110239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c962cc-478b-447c-9105-826962101899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:15.110527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c962cc-478b-447c-9105-826962101899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.112511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c962cc-478b-447c-9105-826962101899 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.133077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1344/2675] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:37:15.135592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.149539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747dd16-c92a-4490-8f18-32cf8c06d708 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.159483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.160131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.160406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.160680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.161480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.163547 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.208785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1332/2676] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:15.217514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c962cc-478b-447c-9105-826962101899 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.229837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c962cc-478b-447c-9105-826962101899 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.337304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.338659 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.339642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.340183 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.341182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.343432 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.386363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1345/2677] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:15 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:15.391566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.412020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ca982-5ff0-4aa9-a6bf-f88f5dab3b35 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.418311 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.419994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.419994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.419994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.420569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.422365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.436060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1333/2678] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:15.442486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.458035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581568f-b5d3-40f7-adde-53f0e8a8c520 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.463780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4afca-a232-49e6-98dd-90f10d0d18f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.464398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4afca-a232-49e6-98dd-90f10d0d18f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.464725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4afca-a232-49e6-98dd-90f10d0d18f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.465026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4afca-a232-49e6-98dd-90f10d0d18f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.465699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4afca-a232-49e6-98dd-90f10d0d18f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.467363 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4afca-a232-49e6-98dd-90f10d0d18f2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.484549 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1346/2679] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:15.489853 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.506437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2d37-d71f-4a6b-81e5-ebecf2225e8a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.512380 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.513164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.513526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.513852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] PATH_INFO: `/v3/projects/335539872e6840bda5ae9db0049078bc/users/24903c0fe11e45848bfd845c8ca8edec/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.543129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] RBAC: Authorizing `identity:create_grant(project_id=335539872e6840bda5ae9db0049078bc, user_id=24903c0fe11e45848bfd845c8ca8edec, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.546516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.571142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1334/2680] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:15 2022] PUT /identity/v3/projects/335539872e6840bda5ae9db0049078bc/users/24903c0fe11e45848bfd845c8ca8edec/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:15.577372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310ef8c-362e-4062-8796-6ad976dad4e3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.578614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310ef8c-362e-4062-8796-6ad976dad4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.579148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310ef8c-362e-4062-8796-6ad976dad4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.602419 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e310ef8c-362e-4062-8796-6ad976dad4e3 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:15.603990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e310ef8c-362e-4062-8796-6ad976dad4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:15.612819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:15.717405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1347/2681] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:15.723438 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:15.725396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:15.807150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0cd77-421a-47c4-a2bc-71e8cb61e4af admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:15.813548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:15.814258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:15.814258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:15.814907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:15.829742 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:15.832722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:15.849551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1335/2682] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:16.956348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:16.956988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:16.957374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:17.032564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] MFA Rules not processed for user `24903c0fe11e45848bfd845c8ca8edec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:17.034763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:17.197705 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1348/2683] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:17.198112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:17.213246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3072f-b166-4f90-bcfe-cf8d2479e08b service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:17.251209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:17.252587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:17.252971 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:17.253533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:17.254771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:17.263767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:17.386730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1336/2684] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:17 2022] POST /identity/v3/projects => generated 371 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:17.391180 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:17.408067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724f405-3daa-4911-9f0a-7eec8e23784d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:17.544356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:17.545041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:17.545349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:17.545627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:17.546297 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:17.548525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:17.588922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1349/2685] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:17 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:37:17.593474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:17.619354 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa17e4c-0a2d-422e-8515-bb225003d57d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:17.627141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:17.628252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:17.629315 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:17.629841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:17.630734 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:17.633515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:17.653983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1337/2686] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:17.658091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:17.678389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a14e23-c8b4-4ec5-afc9-ae9deb965e84 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:17.689076 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:17.689995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:17.690564 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:17.690814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] PATH_INFO: `/v3/projects/a91bf96895184e1c8a880b2d88037cbe/users/98babee43b294ed1ab0bd654f0a3fecf/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:17.724146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91bf96895184e1c8a880b2d88037cbe, user_id=98babee43b294ed1ab0bd654f0a3fecf, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:17.726854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:17.753255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1350/2687] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:17 2022] PUT /identity/v3/projects/a91bf96895184e1c8a880b2d88037cbe/users/98babee43b294ed1ab0bd654f0a3fecf/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:17.757344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:17.784259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3248f-ef6e-4401-ad1e-15da7711f362 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:17.810805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:17.936371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14999d67-fb8d-4ba2-8915-2e42f177681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:17.937150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14999d67-fb8d-4ba2-8915-2e42f177681d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:17.937468 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14999d67-fb8d-4ba2-8915-2e42f177681d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:17.937757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14999d67-fb8d-4ba2-8915-2e42f177681d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:17.938459 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14999d67-fb8d-4ba2-8915-2e42f177681d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:17.940135 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14999d67-fb8d-4ba2-8915-2e42f177681d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:17.955326 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1338/2688] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:17.961391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:17.983150 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3195e5f-4495-4337-ad28-7a9a5ab53a25 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:17.996768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:17.998267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:17.999243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:17.999243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] PATH_INFO: `/v3/projects/a91bf96895184e1c8a880b2d88037cbe/users/98babee43b294ed1ab0bd654f0a3fecf/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:18.004063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91bf96895184e1c8a880b2d88037cbe, user_id=98babee43b294ed1ab0bd654f0a3fecf, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:18.009395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:18.014337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:18.037872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1351/2689] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:17 2022] PUT /identity/v3/projects/a91bf96895184e1c8a880b2d88037cbe/users/98babee43b294ed1ab0bd654f0a3fecf/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:19.044610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72460ce8-6122-4eac-8340-f11215e3932d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:19.045343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72460ce8-6122-4eac-8340-f11215e3932d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:19.045343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72460ce8-6122-4eac-8340-f11215e3932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:19.109691 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-72460ce8-6122-4eac-8340-f11215e3932d None None] MFA Rules not processed for user `98babee43b294ed1ab0bd654f0a3fecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:19.110956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-72460ce8-6122-4eac-8340-f11215e3932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:19.297658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1339/2690] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:19.309958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:19.312021 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:19.454509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f707051-243a-4be4-853c-406856173c95 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:19.460836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:19.461723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:19.462594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:19.463161 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:19.562275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:19.563872 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:19.577209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1352/2691] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:37:20.542806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239faae1-1c11-4fab-8589-b98847007839 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:20.544009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239faae1-1c11-4fab-8589-b98847007839 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:20.544009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239faae1-1c11-4fab-8589-b98847007839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:20.575677 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-239faae1-1c11-4fab-8589-b98847007839 None None] MFA Rules not processed for user `c1175476deaf4f14967accc136a0b823`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:20.576903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-239faae1-1c11-4fab-8589-b98847007839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:20.658666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1340/2692] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:20.679019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:20.680529 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:20.787865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:20.797598 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a715b-c080-4377-9f65-567df1fe2dcd service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:20.804948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:20.807206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:20.807206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:20.807206 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:20.809948 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:20.878347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:20.879805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:20.895627 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1353/2693] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:21.131561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239faae1-1c11-4fab-8589-b98847007839 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:21.133805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-239faae1-1c11-4fab-8589-b98847007839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:21.155467 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239faae1-1c11-4fab-8589-b98847007839 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:21.165790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:21.166427 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:21.168134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:21.168423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:21.190931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:21.192875 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:21.211013 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1341/2694] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:30.885923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=28 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:31.201668 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:34.685567 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:34.687387 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:34.792519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559f9b3-0ae9-476a-bb6e-94761bc6770f service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:34.797836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:34.798569 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:34.798845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:34.799122 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:34.801041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:34.813603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:34.815713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:34.826696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1354/2695] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:35.270309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:35.345394 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f80870-1f71-4c89-92ff-3190ce9447d9 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:35.350826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:35.351499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:35.351774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:35.352065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] PATH_INFO: `/v3/users/24903c0fe11e45848bfd845c8ca8edec` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:35.353788 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=24903c0fe11e45848bfd845c8ca8edec)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:35.355762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:35.414976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24903c0fe11e45848bfd845c8ca8edec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.424141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24903c0fe11e45848bfd845c8ca8edec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.424944 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24903c0fe11e45848bfd845c8ca8edec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.425758 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24903c0fe11e45848bfd845c8ca8edec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.435843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24903c0fe11e45848bfd845c8ca8edec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.479248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1342/2696] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:35 2022] DELETE /identity/v3/users/24903c0fe11e45848bfd845c8ca8edec => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:35.694131 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:35.706594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f87afa-dd19-4f86-98a0-c32c44f045d6 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:35.773156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:35.773790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:35.774069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:35.774330 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] PATH_INFO: `/v3/projects/335539872e6840bda5ae9db0049078bc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:35.775527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=335539872e6840bda5ae9db0049078bc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:35.777466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:35.829231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335539872e6840bda5ae9db0049078bc', 'request_id': 'req-a7f2c5ed-8a11-411c-b916-dcf8aae91305'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.835994 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335539872e6840bda5ae9db0049078bc', 'request_id': 'req-a7f2c5ed-8a11-411c-b916-dcf8aae91305'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.845239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335539872e6840bda5ae9db0049078bc', 'request_id': 'req-a7f2c5ed-8a11-411c-b916-dcf8aae91305'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.854423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:35.859596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] The token cache is being invalidate because project 335539872e6840bda5ae9db0049078bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:35.860148 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.861688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:35.864398 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1355/2697] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:35 2022] DELETE /identity/v3/projects/335539872e6840bda5ae9db0049078bc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:35.868623 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:35.870150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:35.883688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbd469-a3ec-4614-bc04-44e3eb321cec admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:35.944597 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:35.945289 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:35.945557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:35.945866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] PATH_INFO: `/v3/users/98babee43b294ed1ab0bd654f0a3fecf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:35.947227 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98babee43b294ed1ab0bd654f0a3fecf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:35.949228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:35.974886 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:36.006493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98babee43b294ed1ab0bd654f0a3fecf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.014859 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98babee43b294ed1ab0bd654f0a3fecf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.015484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98babee43b294ed1ab0bd654f0a3fecf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.016217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98babee43b294ed1ab0bd654f0a3fecf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.024849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-171236f1-b691-42ab-ba33-474b12a22127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98babee43b294ed1ab0bd654f0a3fecf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.043208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1343/2698] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:35 2022] DELETE /identity/v3/users/98babee43b294ed1ab0bd654f0a3fecf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:36.230067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.231778 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:36.248789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2c5ed-8a11-411c-b916-dcf8aae91305 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.312685 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.313493 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.313940 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.314490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] PATH_INFO: `/v3/projects/a91bf96895184e1c8a880b2d88037cbe` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.316982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a91bf96895184e1c8a880b2d88037cbe)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.319555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.383764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a91bf96895184e1c8a880b2d88037cbe', 'request_id': 'req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.390556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a91bf96895184e1c8a880b2d88037cbe', 'request_id': 'req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.399490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a91bf96895184e1c8a880b2d88037cbe', 'request_id': 'req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.417766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] The token cache is being invalidate because project a91bf96895184e1c8a880b2d88037cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:36.418349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.419445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fd4fb97-b63b-4ee9-8194-dc7aa67b849c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:36.422244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1356/2699] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:36 2022] DELETE /identity/v3/projects/a91bf96895184e1c8a880b2d88037cbe => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:36.427189 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.427671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.427924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.448605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:36.449898 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:36.527593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1344/2700] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:36.535829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.536291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.536698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.559234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:36.560502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:36.568519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1357/2701] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:37:36.573516 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.588419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb53d99-0af0-42fb-8d83-4a19dab3b66b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.593091 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.593689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.593959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.594223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.594876 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:36.595175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.596766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.610701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1345/2702] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:37:36.615496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.627906 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaea631-945d-4de3-a967-8a5beb5fcc03 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.632689 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.633935 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.634264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.634538 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.635190 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.637138 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.676155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1358/2703] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:36 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:36.679700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.696661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a81c7-0563-4183-9057-a1e2f9cc9cdd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.793170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.793915 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.794352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.794724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.795499 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.797868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.821411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1346/2704] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:36 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:36.825563 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.835944 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232b979-b2c2-4019-9bc3-b386a075cc85 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.841617 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.842184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.842637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.842939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.844107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.846048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.856934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1359/2705] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:36.861147 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.871935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63462c5-a0e7-44e1-9576-55a8e314b4d2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.876572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4047a0-09b4-4189-8b0c-e9118b2a2a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.877198 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4047a0-09b4-4189-8b0c-e9118b2a2a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.877473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4047a0-09b4-4189-8b0c-e9118b2a2a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.877741 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4047a0-09b4-4189-8b0c-e9118b2a2a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.878384 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4047a0-09b4-4189-8b0c-e9118b2a2a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.879977 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4047a0-09b4-4189-8b0c-e9118b2a2a4f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.892363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1347/2706] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:36.897618 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:36.908686 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f839a5-0d6a-4dfe-8e1a-20a471157f45 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:36.915202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:36.915846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.916114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.916404 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] PATH_INFO: `/v3/projects/fcd23d9a231f4cd0a244647ea6c9cb98/users/48590097f83b4f9ebcaa21341358c7ab/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.935730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd23d9a231f4cd0a244647ea6c9cb98, user_id=48590097f83b4f9ebcaa21341358c7ab, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:36.938718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:36.963706 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1360/2707] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:36 2022] PUT /identity/v3/projects/fcd23d9a231f4cd0a244647ea6c9cb98/users/48590097f83b4f9ebcaa21341358c7ab/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:36.965981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:36.966332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:36.966552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:36.987254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:37.011970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] MFA Rules not processed for user `48590097f83b4f9ebcaa21341358c7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:37.013180 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:37.115173 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1348/2708] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:37.116237 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.116633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.118266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:37.159184 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:37.213635 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ebb28-ffdf-4131-a6c4-99ca0557faa5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.218527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.218669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcd145-ccad-4eb6-96ce-87456e5f1dba None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.219142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.219422 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.219719 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.220348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.222313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.225724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.226130 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.226470 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.226882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.228708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:37.242108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.244031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.255762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1361/2709] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:37 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:37.256378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1349/2710] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:37.260371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.336950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604da0ad-c592-4f9b-8582-82ccf03e4dea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.344889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.345550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.345825 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.346088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.346724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.348692 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.377818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1362/2711] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:37 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:37.378998 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.391393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a0e4-e7ef-434a-8d6d-10affda19f86 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.396348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.396983 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.397282 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.397552 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.398191 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.400234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.413070 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1350/2712] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:37.416467 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.431095 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd27c99-b255-4747-bee2-1f7dabea5de4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.438316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40357e6f-e196-43f5-9687-71675a5206f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.439045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40357e6f-e196-43f5-9687-71675a5206f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.439361 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40357e6f-e196-43f5-9687-71675a5206f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.439632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40357e6f-e196-43f5-9687-71675a5206f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.440323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40357e6f-e196-43f5-9687-71675a5206f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.441998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40357e6f-e196-43f5-9687-71675a5206f1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.457386 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1363/2713] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:37.463291 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.483316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1aa56-7f1e-4964-824d-e7b5a4905969 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.488323 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.489120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.489462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.489805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] PATH_INFO: `/v3/projects/7f201af1477942fe8438839ab03db462/users/ec23338bb18549c1986b23d0d62aaaab/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.509378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f201af1477942fe8438839ab03db462, user_id=ec23338bb18549c1986b23d0d62aaaab, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.512699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.529293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1351/2714] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:37 2022] PUT /identity/v3/projects/7f201af1477942fe8438839ab03db462/users/ec23338bb18549c1986b23d0d62aaaab/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:37:37.599677 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.600065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.600294 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.630521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] MFA Rules not processed for user `ec23338bb18549c1986b23d0d62aaaab`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:37.631806 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:37.718005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1364/2715] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:37.723576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.811141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:37.833334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b7d4-987e-4da2-b6b3-795965cf7a7a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.840919 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.841679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.842075 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.842392 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.843154 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.845922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:37.882101 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1352/2716] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:37 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:37.891951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:37.909818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23bac1-6fe6-457e-b707-90c859e20748 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:37.939514 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:37.991735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:37.992487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:37.992768 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:37.993054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:37.993736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:37.995755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:38.018579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1365/2717] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:37 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:38.022679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:38.034960 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467891-7fa5-486a-a870-9b528f8f5ffa admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:38.040092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:38.040700 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.040981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.041285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.041937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:38.043561 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:38.057037 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1353/2718] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:38.061204 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:38.071774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7133a-8706-4524-8b06-1fc520da917d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:38.076639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:38.077320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.077594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.077888 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] PATH_INFO: `/v3/projects/c7dd703aed044ed6b09dbc73315e6548/users/7fbd75d85de54ec2a1aa0aff50306476/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.093570 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7dd703aed044ed6b09dbc73315e6548, user_id=7fbd75d85de54ec2a1aa0aff50306476, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:38.096402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:38.110937 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1366/2719] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:38 2022] PUT /identity/v3/projects/c7dd703aed044ed6b09dbc73315e6548/users/7fbd75d85de54ec2a1aa0aff50306476/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:38.114743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:38.126453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001fd67-f646-40d3-be33-472a5fe110e2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:38.323118 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a827c56-be9c-4632-ada6-47f35f718d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:38.323769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827c56-be9c-4632-ada6-47f35f718d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.324046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827c56-be9c-4632-ada6-47f35f718d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.324319 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827c56-be9c-4632-ada6-47f35f718d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.324989 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827c56-be9c-4632-ada6-47f35f718d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:38.326662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827c56-be9c-4632-ada6-47f35f718d63 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:38.340620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1354/2720] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:38.341847 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:38.352510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f477b-3738-4a6c-a70b-fc9531372e0c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:38.357664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:38.358318 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.358577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.358850 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] PATH_INFO: `/v3/projects/c7dd703aed044ed6b09dbc73315e6548/users/7fbd75d85de54ec2a1aa0aff50306476/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.360822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7dd703aed044ed6b09dbc73315e6548, user_id=7fbd75d85de54ec2a1aa0aff50306476, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:38.363454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:38.374774 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1367/2721] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:38 2022] PUT /identity/v3/projects/c7dd703aed044ed6b09dbc73315e6548/users/7fbd75d85de54ec2a1aa0aff50306476/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:38.385382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089c3c8-c584-4159-962a-b68f7f445334 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.385664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089c3c8-c584-4159-962a-b68f7f445334 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.385934 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089c3c8-c584-4159-962a-b68f7f445334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.412982 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1089c3c8-c584-4159-962a-b68f7f445334 None None] MFA Rules not processed for user `7fbd75d85de54ec2a1aa0aff50306476`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:38.414709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1089c3c8-c584-4159-962a-b68f7f445334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:38.482803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1355/2722] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:38.494030 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:38.495716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:38.552328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609fef1-577b-4ea9-84a7-0153bf694262 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:38.556060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:38.556650 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.556921 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.557225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.613377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:38.614805 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:38.624646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1368/2723] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:38.730596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:38.731067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:38.731377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:38.753210 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] MFA Rules not processed for user `ff4810b8d4ef4c80a47b3208ce5439d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:38.754423 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:38.798785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:38.815583 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1356/2724] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:40.183166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:40.184620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:40.195827 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704daad-4c75-4883-9bf2-1c25b29158cb service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:40.199468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:40.200058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:40.200331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:40.200594 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:40.227582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:40.262960 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:40.264378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:40.273496 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1369/2725] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:40.835697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:40.846672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cc57b-ac59-4c0f-ae19-efacfb8d5b0c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:40.850484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c2520-ea8f-43ad-adb4-dcf9a0ce2331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:40.851094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c2520-ea8f-43ad-adb4-dcf9a0ce2331 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:40.851370 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c2520-ea8f-43ad-adb4-dcf9a0ce2331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:40.851671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c2520-ea8f-43ad-adb4-dcf9a0ce2331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:40.861163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c2520-ea8f-43ad-adb4-dcf9a0ce2331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:40.862593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c2520-ea8f-43ad-adb4-dcf9a0ce2331 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:40.873300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1357/2726] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:44.827871 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:44.901506 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51567bf-647e-4b90-bcc9-bff0f8910653 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:44.906143 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:44.906763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:44.907033 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:44.907298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] PATH_INFO: `/v3/users/48590097f83b4f9ebcaa21341358c7ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:44.908446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48590097f83b4f9ebcaa21341358c7ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:44.910352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:44.959613 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48590097f83b4f9ebcaa21341358c7ab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:44.967480 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48590097f83b4f9ebcaa21341358c7ab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:44.968410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48590097f83b4f9ebcaa21341358c7ab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:44.969243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48590097f83b4f9ebcaa21341358c7ab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:44.976503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48590097f83b4f9ebcaa21341358c7ab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:44.999591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1370/2727] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:44 2022] DELETE /identity/v3/users/48590097f83b4f9ebcaa21341358c7ab => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:45.004094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:45.004558 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:45.004961 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:45.035970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:45.037517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:45.112980 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1358/2728] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:45.129854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:45.135959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:45.200713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:45.217105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f3e41-f9ea-4860-a61e-1edc1a59c254 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:45.220756 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:45.221509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:45.221889 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:45.222268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:45.232452 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:45.234079 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:45.244642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1371/2729] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:45.572917 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:45.575376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:45.595781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bca4e-876f-4f50-8e21-1a547d195d6f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:45.602164 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:45.602864 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:45.603256 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:45.603605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] PATH_INFO: `/v3/projects/fcd23d9a231f4cd0a244647ea6c9cb98` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:45.605714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd23d9a231f4cd0a244647ea6c9cb98)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:45.607717 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:45.670409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd23d9a231f4cd0a244647ea6c9cb98', 'request_id': 'req-660b4162-2beb-4e1c-a34a-b4b3537551e6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.677466 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd23d9a231f4cd0a244647ea6c9cb98', 'request_id': 'req-660b4162-2beb-4e1c-a34a-b4b3537551e6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.687345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd23d9a231f4cd0a244647ea6c9cb98', 'request_id': 'req-660b4162-2beb-4e1c-a34a-b4b3537551e6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.699852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] The token cache is being invalidate because project fcd23d9a231f4cd0a244647ea6c9cb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:45.700161 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.700866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.702454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1359/2730] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:45 2022] DELETE /identity/v3/projects/fcd23d9a231f4cd0a244647ea6c9cb98 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:45.706488 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:45.718981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c33c-b9e5-4735-a827-ba530592e690 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:45.791560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:45.792327 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:45.792530 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:45.792771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] PATH_INFO: `/v3/users/ec23338bb18549c1986b23d0d62aaaab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:45.794066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec23338bb18549c1986b23d0d62aaaab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:45.795970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:45.854182 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec23338bb18549c1986b23d0d62aaaab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.862710 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec23338bb18549c1986b23d0d62aaaab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.863674 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec23338bb18549c1986b23d0d62aaaab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.864307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec23338bb18549c1986b23d0d62aaaab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.874814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec23338bb18549c1986b23d0d62aaaab'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:45.898664 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1372/2731] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:45 2022] DELETE /identity/v3/users/ec23338bb18549c1986b23d0d62aaaab => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:46.265743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:46.267471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:46.285591 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b4162-2beb-4e1c-a34a-b4b3537551e6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:46.351828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:46.352567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:46.352857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:46.353170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] PATH_INFO: `/v3/projects/7f201af1477942fe8438839ab03db462` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:46.354709 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f201af1477942fe8438839ab03db462)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:46.356785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:46.358607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:46.416341 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f201af1477942fe8438839ab03db462', 'request_id': 'req-3990b7cc-7452-48b1-b116-6393b9875f26'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.423605 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f201af1477942fe8438839ab03db462', 'request_id': 'req-3990b7cc-7452-48b1-b116-6393b9875f26'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.432421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f201af1477942fe8438839ab03db462', 'request_id': 'req-3990b7cc-7452-48b1-b116-6393b9875f26'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.443743 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] The token cache is being invalidate because project 7f201af1477942fe8438839ab03db462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:46.444045 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.444678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.446744 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1360/2732] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:46 2022] DELETE /identity/v3/projects/7f201af1477942fe8438839ab03db462 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:46.450699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:46.452246 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:46.465331 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72150aa-132a-4d2c-a111-f34786df69b5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:46.525445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:46.526105 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:46.526379 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:46.526645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] PATH_INFO: `/v3/users/7fbd75d85de54ec2a1aa0aff50306476` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:46.527791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fbd75d85de54ec2a1aa0aff50306476)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:46.529858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:46.536797 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:46.586995 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fbd75d85de54ec2a1aa0aff50306476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.594989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fbd75d85de54ec2a1aa0aff50306476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.595740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fbd75d85de54ec2a1aa0aff50306476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.596380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fbd75d85de54ec2a1aa0aff50306476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.604225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47e1a583-1dd9-41e1-8b5d-77930de42481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fbd75d85de54ec2a1aa0aff50306476'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:46.623064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1373/2733] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:37:46 2022] DELETE /identity/v3/users/7fbd75d85de54ec2a1aa0aff50306476 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:47.000040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.001489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:47.018368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990b7cc-7452-48b1-b116-6393b9875f26 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.078737 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.079371 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.079646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.079955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] PATH_INFO: `/v3/projects/c7dd703aed044ed6b09dbc73315e6548` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.081113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7dd703aed044ed6b09dbc73315e6548)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.083028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.135833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7dd703aed044ed6b09dbc73315e6548', 'request_id': 'req-042e3fcc-1416-4095-a09a-8251f03e1a2c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:47.146769 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7dd703aed044ed6b09dbc73315e6548', 'request_id': 'req-042e3fcc-1416-4095-a09a-8251f03e1a2c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:47.156809 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7dd703aed044ed6b09dbc73315e6548', 'request_id': 'req-042e3fcc-1416-4095-a09a-8251f03e1a2c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:47.168548 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] The token cache is being invalidate because project c7dd703aed044ed6b09dbc73315e6548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:37:47.168785 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:47.169447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-042e3fcc-1416-4095-a09a-8251f03e1a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:37:47.171122 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1361/2734] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:37:46 2022] DELETE /identity/v3/projects/c7dd703aed044ed6b09dbc73315e6548 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:47.175848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.176390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.176499 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.196867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:47.198130 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:47.260089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1374/2735] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:47.267877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.268374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.268478 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.288683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:47.290554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:47.298229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1362/2736] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:47.304091 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.313859 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250db38a-4377-4224-b164-e9d118ee8b3e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.318476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.319074 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.319351 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.319624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.320288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:37:47.320593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.322221 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.333967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1375/2737] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:37:47.338834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.348132 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021422-e27d-40a7-b8a1-e1de46245e1b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.353452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.354035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.354308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.354575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.355196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.357194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.368178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:47.386827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1363/2738] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:37:47 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:47.391277 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.401899 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31e8e8-6847-440c-9c83-51f840e8fc23 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.459884 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.460482 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.461406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.461684 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.462325 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.464866 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.486093 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1376/2739] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:37:47 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:37:47.490136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.500444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45cee-0bc8-4323-a7f6-75eebba36165 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.505292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.505913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.506190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.506458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.507100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.509217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.519251 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1364/2740] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:47.523877 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.529265 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:37:47.535836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b75b4-333c-4bbe-afd0-0d44d7498dcd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.541747 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2304ce5-810c-4262-85e1-9137ae8800e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.542372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2304ce5-810c-4262-85e1-9137ae8800e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.542670 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2304ce5-810c-4262-85e1-9137ae8800e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.542992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2304ce5-810c-4262-85e1-9137ae8800e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.543669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2304ce5-810c-4262-85e1-9137ae8800e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.545355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2304ce5-810c-4262-85e1-9137ae8800e2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.556146 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1377/2741] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:37:47.560310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.570812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9dc96-d4de-44d7-9a9e-8ed949ac2896 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.575517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.576153 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.576431 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.576704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] PATH_INFO: `/v3/projects/99e7b617783e468797748aed49178f47/users/19f7d08f087d4bd194882999a04f97d4/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.594382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e7b617783e468797748aed49178f47, user_id=19f7d08f087d4bd194882999a04f97d4, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.597068 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.609835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1365/2742] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:37:47 2022] PUT /identity/v3/projects/99e7b617783e468797748aed49178f47/users/19f7d08f087d4bd194882999a04f97d4/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:37:47.620157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c406d-8cce-4f92-8818-36588caacbf1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.620448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c406d-8cce-4f92-8818-36588caacbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.620723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c406d-8cce-4f92-8818-36588caacbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.650483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-465c406d-8cce-4f92-8818-36588caacbf1 None None] MFA Rules not processed for user `19f7d08f087d4bd194882999a04f97d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:47.651761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-465c406d-8cce-4f92-8818-36588caacbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:47.718443 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1378/2743] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:47.730352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:47.732097 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:47.787508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d7dd-54c5-4dc3-aa59-4d03776081ba admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:47.791678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:47.792276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:47.792553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:47.792830 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:47.801080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:47.802500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:47.811680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1366/2744] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:49.247356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:49.247926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:49.247926 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:49.266657 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] MFA Rules not processed for user `a73941feee334ac294ac875001b148fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:37:49.267845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:49.331612 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1379/2745] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:37:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:37:57.111844 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:37:57.113386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:57.172651 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9c07e-3f03-4432-bf2e-610513a3a56a service glance] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:37:57.176216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:37:57.176835 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:37:57.177141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:37:57.177406 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:37:57.179084 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:37:57.245036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:37:57.246757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda6df8-7ab9-4191-85fc-aa3c17c6b0d0 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:37:57.257046 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1367/2746] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:37:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:37:59.320620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=61 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:07.252957 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:12.629478 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:12.694078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb9da2-8080-4a29-99eb-e4caaf892ed4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:12.699167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:12.699787 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:12.700055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:12.700320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] PATH_INFO: `/v3/users/19f7d08f087d4bd194882999a04f97d4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:12.701519 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f7d08f087d4bd194882999a04f97d4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:12.703426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:12.750867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f7d08f087d4bd194882999a04f97d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:12.758275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f7d08f087d4bd194882999a04f97d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:12.758911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f7d08f087d4bd194882999a04f97d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:12.759520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f7d08f087d4bd194882999a04f97d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:12.767310 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f7d08f087d4bd194882999a04f97d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:12.785811 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1380/2747] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:38:12 2022] DELETE /identity/v3/users/19f7d08f087d4bd194882999a04f97d4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:38:12.790913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:12.791434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:12.791434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:12.810925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:38:12.812125 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:12.872079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1368/2748] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:38:12.882840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:12.885209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:12.943264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502debf4-ecf7-4a19-81d3-f47e166c7368 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:12.947444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c615400-2d99-4d5e-bbec-3c2a7ff20083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:12.948043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c615400-2d99-4d5e-bbec-3c2a7ff20083 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:12.948323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c615400-2d99-4d5e-bbec-3c2a7ff20083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:12.948601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c615400-2d99-4d5e-bbec-3c2a7ff20083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:12.957712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c615400-2d99-4d5e-bbec-3c2a7ff20083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:12.959210 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c615400-2d99-4d5e-bbec-3c2a7ff20083 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:12.969369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1381/2749] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 12:38:13.239802 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.242067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:13.260471 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc397b-ff67-44b3-a5cb-34bafb3a5051 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.265369 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.266004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.266275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.266538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] PATH_INFO: `/v3/projects/99e7b617783e468797748aed49178f47` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.267829 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e7b617783e468797748aed49178f47)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.270079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.318284 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e7b617783e468797748aed49178f47', 'request_id': 'req-7c190fd4-0f69-4a03-bfdc-f6c875754338'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:13.325184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e7b617783e468797748aed49178f47', 'request_id': 'req-7c190fd4-0f69-4a03-bfdc-f6c875754338'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:13.333962 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e7b617783e468797748aed49178f47', 'request_id': 'req-7c190fd4-0f69-4a03-bfdc-f6c875754338'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:13.344895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] The token cache is being invalidate because project 99e7b617783e468797748aed49178f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:38:13.345216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:13.345913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c190fd4-0f69-4a03-bfdc-f6c875754338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:13.347414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1369/2750] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:38:13 2022] DELETE /identity/v3/projects/99e7b617783e468797748aed49178f47 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:38:13.352423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.352707 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.352981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.375126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:38:13.376356 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:13.397366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:13.443222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1382/2751] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:38:13.451873 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.452186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.452482 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.473747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:38:13.474954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:13.483408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1370/2752] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:38:13.487955 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.498828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62952fae-8751-4c2c-9f9d-fee9cc68e616 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.503517 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.504117 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.504395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.504665 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.505342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:38:13.505645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.507239 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.521170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1383/2753] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:38:13.525559 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.535321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8df46-68de-4b7c-ba14-8f138fe2b0b7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.540052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.540694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.540967 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.541261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.541907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.543799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.574890 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1371/2754] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:38:13 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:38:13.579073 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.595660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850702e-bdd8-49be-aeaf-a9e617cb6545 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.650879 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.651588 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.651746 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.652013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.652669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.654606 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.676737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1384/2755] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:38:13 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:38:13.681602 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.699697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03e10d-8e3e-4443-8f9a-ca6b261163a9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.707515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.708526 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.709088 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.709594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.710799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.714632 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.734416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1372/2756] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:38:13.738466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.749054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba42d-e236-41fc-a560-47f36b0ffd69 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.753758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223a6b-8d22-4b51-a59b-28cbb46a335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.754264 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223a6b-8d22-4b51-a59b-28cbb46a335b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.754537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223a6b-8d22-4b51-a59b-28cbb46a335b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.754803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223a6b-8d22-4b51-a59b-28cbb46a335b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.755448 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e223a6b-8d22-4b51-a59b-28cbb46a335b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.757248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e223a6b-8d22-4b51-a59b-28cbb46a335b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.768711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1385/2757] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:38:13.773181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.784036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8288d-3ac6-4f0a-9e2c-51bc417d21e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:13.788753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:13.789435 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.789733 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.790003 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] PATH_INFO: `/v3/projects/12bbdb19ca3348c7b7f7c15b141b4e47/users/7026da5349b84b76b38a0749841a4200/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.805151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bbdb19ca3348c7b7f7c15b141b4e47, user_id=7026da5349b84b76b38a0749841a4200, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:13.807777 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:13.823576 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1373/2758] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:38:13 2022] PUT /identity/v3/projects/12bbdb19ca3348c7b7f7c15b141b4e47/users/7026da5349b84b76b38a0749841a4200/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:38:13.835420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:13.835646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:13.835917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:13.869688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] MFA Rules not processed for user `7026da5349b84b76b38a0749841a4200`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:38:13.871169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:13.939254 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1386/2759] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:38:13.950912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:13.952663 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:13.976233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:14.016762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba674b2-8e58-4792-ae12-cc5e989d38f9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:14.020572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af82d55-69b4-4240-be1a-e119af6c3935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:14.021208 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af82d55-69b4-4240-be1a-e119af6c3935 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:14.021522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af82d55-69b4-4240-be1a-e119af6c3935 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:14.021748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af82d55-69b4-4240-be1a-e119af6c3935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:14.035375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af82d55-69b4-4240-be1a-e119af6c3935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:14.036827 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af82d55-69b4-4240-be1a-e119af6c3935 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:14.047266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1374/2760] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:38:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:38:23.936326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:24.042523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=17 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:24.061745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:24.127614 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052892a-1b88-4fa8-8f43-f15dd5a60985 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:24.132104 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:24.132736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:24.133092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:24.133552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] PATH_INFO: `/v3/users/7026da5349b84b76b38a0749841a4200` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:24.134682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7026da5349b84b76b38a0749841a4200)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:24.136582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:24.189491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7026da5349b84b76b38a0749841a4200'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.198083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7026da5349b84b76b38a0749841a4200'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.198880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7026da5349b84b76b38a0749841a4200'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.199491 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7026da5349b84b76b38a0749841a4200'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.207114 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7026da5349b84b76b38a0749841a4200'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.227716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1387/2761] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:38:24 2022] DELETE /identity/v3/users/7026da5349b84b76b38a0749841a4200 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:38:24.232344 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:24.232617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:24.232885 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:24.253612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:38:24.254826 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:24.314572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1375/2762] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:38:24.324891 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:24.326368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:24.384154 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60398-d47f-451e-ac7f-4fa22beca754 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:24.387897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115ccf9-e0b2-4bee-aa14-5c6b1f445bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:24.388497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115ccf9-e0b2-4bee-aa14-5c6b1f445bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:24.388771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115ccf9-e0b2-4bee-aa14-5c6b1f445bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:24.389152 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115ccf9-e0b2-4bee-aa14-5c6b1f445bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:24.398045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115ccf9-e0b2-4bee-aa14-5c6b1f445bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:24.400059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115ccf9-e0b2-4bee-aa14-5c6b1f445bc7 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:24.409476 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1388/2763] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:38:24.685260 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:38:24.686694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:24.700245 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84629a8e-89a3-44ae-9631-1ec018e2f120 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:38:24.704923 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:38:24.705727 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:24.706077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:24.706353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] PATH_INFO: `/v3/projects/12bbdb19ca3348c7b7f7c15b141b4e47` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:24.707699 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12bbdb19ca3348c7b7f7c15b141b4e47)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:38:24.709664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:38:24.763249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12bbdb19ca3348c7b7f7c15b141b4e47', 'request_id': 'req-78878438-bf81-41ad-b202-391f7009b4a3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.769507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12bbdb19ca3348c7b7f7c15b141b4e47', 'request_id': 'req-78878438-bf81-41ad-b202-391f7009b4a3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.777146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12bbdb19ca3348c7b7f7c15b141b4e47', 'request_id': 'req-78878438-bf81-41ad-b202-391f7009b4a3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.788102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] The token cache is being invalidate because project 12bbdb19ca3348c7b7f7c15b141b4e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:38:24.788386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.789031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:38:24.790497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1376/2764] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:38:24 2022] DELETE /identity/v3/projects/12bbdb19ca3348c7b7f7c15b141b4e47 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:38:34.405283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:34.785197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:38:48.135711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:38:48.136275 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:38:48.136384 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:38:48.156854 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] MFA Rules not processed for user `a73941feee334ac294ac875001b148fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:38:48.158112 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:38:48.215395 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1389/2765] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:38:58.207970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:36.357232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:36.358683 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:36.417081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78878438-bf81-41ad-b202-391f7009b4a3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:36.420988 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:36.421619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:36.421896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:36.422168 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:36.423916 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:36.497185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:36.498593 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:36.511057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1377/2766] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:39:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:39:43.385454 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:43.395636 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565febe-d6d1-4050-83e7-14ad81a85d71 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:43.399189 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:43.399775 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:43.400052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:43.400322 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:43.402068 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:43.416603 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:43.418113 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:43.431299 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1390/2767] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:39:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:39:46.506155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=54 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:53.427445 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:55.930041 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:55.931566 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:56.006290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8b2fe-f419-47ff-bcac-4b3635327c58 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.011242 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.011869 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.012142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.012413 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] PATH_INFO: `/v3/users/a73941feee334ac294ac875001b148fb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.013596 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73941feee334ac294ac875001b148fb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.015510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.062048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73941feee334ac294ac875001b148fb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.069724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73941feee334ac294ac875001b148fb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.070353 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73941feee334ac294ac875001b148fb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.071043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73941feee334ac294ac875001b148fb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.078231 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-00c4bc1d-2f1f-4e8c-a157-68fc983305ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73941feee334ac294ac875001b148fb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.096659 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1378/2768] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:39:55 2022] DELETE /identity/v3/users/a73941feee334ac294ac875001b148fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:39:56.268765 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.280938 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f20754-1a05-4046-96d9-260337602024 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.342998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.343621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.343893 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.344158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] PATH_INFO: `/v3/projects/3a32a1e0f4ae442398d6e179846e1e1b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.345391 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a32a1e0f4ae442398d6e179846e1e1b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.347267 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.394224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a32a1e0f4ae442398d6e179846e1e1b', 'request_id': 'req-90481b9b-821a-4ea1-871e-d71ab223ab45'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.400851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a32a1e0f4ae442398d6e179846e1e1b', 'request_id': 'req-90481b9b-821a-4ea1-871e-d71ab223ab45'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.408394 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a32a1e0f4ae442398d6e179846e1e1b', 'request_id': 'req-90481b9b-821a-4ea1-871e-d71ab223ab45'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.418767 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] The token cache is being invalidate because project 3a32a1e0f4ae442398d6e179846e1e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:39:56.419047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.419673 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90481b9b-821a-4ea1-871e-d71ab223ab45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:39:56.421167 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1391/2769] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:39:56 2022] DELETE /identity/v3/projects/3a32a1e0f4ae442398d6e179846e1e1b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:39:56.426073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.426352 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.426620 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.447308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:39:56.448491 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:56.506938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1379/2770] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:39:56.515453 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.515737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.516006 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.536735 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:39:56.537990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:56.546177 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1392/2771] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:39:56.550607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.560316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e0a4d-9960-47d2-b5ee-0429ef0c5e41 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.564902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.565520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.565798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.566599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.567233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:39:56.567530 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.569129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.580447 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1380/2772] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:39:56.585420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.594890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc9b9b-f358-4c2d-8af9-4a1eb7345343 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.599602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.600185 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.600455 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.600717 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.601377 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.603245 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.640273 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1393/2773] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:39:56 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:39:56.644679 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.655368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365a213-b6b7-4b66-a4b5-43bebd4933ca admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.658994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:56.718436 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.719040 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.719309 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.719569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.720270 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.722416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.746338 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1381/2774] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:39:56 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:39:56.751702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.763208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b657d-380b-42ad-a450-41fffef6e303 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.768106 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.768725 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.769544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.769829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.770520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.772207 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.782930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1394/2775] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:39:56.787666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.797840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096528-c37e-4f84-bacc-f32b1183c009 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.802520 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98ac1-ce30-4646-a070-e2773b0fe3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.803112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98ac1-ce30-4646-a070-e2773b0fe3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.803379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98ac1-ce30-4646-a070-e2773b0fe3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.803642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98ac1-ce30-4646-a070-e2773b0fe3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.804274 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98ac1-ce30-4646-a070-e2773b0fe3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.805878 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98ac1-ce30-4646-a070-e2773b0fe3e8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.823259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1382/2776] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:39:56.827433 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:56.837958 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5ea4f-19fb-4947-8d37-98cc6f8ae9a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:56.842518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:56.843149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.843426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.843693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] PATH_INFO: `/v3/projects/df34e34dcb374c41bab3a990a2cf2ea7/users/8e0c1ce44f8e475d95549802b4dca2a8/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.858592 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34e34dcb374c41bab3a990a2cf2ea7, user_id=8e0c1ce44f8e475d95549802b4dca2a8, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:56.861258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:56.874020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1395/2777] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:39:56 2022] PUT /identity/v3/projects/df34e34dcb374c41bab3a990a2cf2ea7/users/8e0c1ce44f8e475d95549802b4dca2a8/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:39:56.884924 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:56.885321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:56.885592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:56.913954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] MFA Rules not processed for user `8e0c1ce44f8e475d95549802b4dca2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:39:56.915079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:56.979082 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1383/2778] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:39:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:39:56.983568 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.002821 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:57.050755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbb01c-0c9b-491e-a355-25c0746f2acb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.055408 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.056002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.056291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.056557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.057208 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.059070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.086979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1396/2779] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:39:56 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:39:57.091026 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.102487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298f7c9-5374-48cf-8a00-8630e5b0bfd7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.170947 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.171587 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.171856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.172127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.172749 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.174913 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.200500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1384/2780] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:39:57 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:39:57.204724 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.215777 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66977c85-0786-4893-b4bd-3d6bd2c9b38b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.220864 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.221582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.221816 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.222111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.222814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.224495 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.236307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1397/2781] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:39:57.241451 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.248560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:57.254379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621adb21-f61f-4459-aad7-016bcd7e20f6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.260887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.261603 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.261868 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.262150 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] PATH_INFO: `/v3/projects/17b8531a16c4491a8cd0737d7f11b126/users/7bd15fc30e3148ddaf13fc747bf1e3cd/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.278571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] RBAC: Authorizing `identity:create_grant(project_id=17b8531a16c4491a8cd0737d7f11b126, user_id=7bd15fc30e3148ddaf13fc747bf1e3cd, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.281267 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.296524 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1385/2782] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:39:57 2022] PUT /identity/v3/projects/17b8531a16c4491a8cd0737d7f11b126/users/7bd15fc30e3148ddaf13fc747bf1e3cd/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:39:57.301605 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.312675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2514a-4fa6-4b68-85c7-93e1654fa70a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.374276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2497184-f555-4baf-8a9f-b5da55a7af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.374969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2497184-f555-4baf-8a9f-b5da55a7af12 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.375251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2497184-f555-4baf-8a9f-b5da55a7af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.375521 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2497184-f555-4baf-8a9f-b5da55a7af12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.376166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2497184-f555-4baf-8a9f-b5da55a7af12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.377793 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2497184-f555-4baf-8a9f-b5da55a7af12 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.388257 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1398/2783] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:39:57.392647 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.403036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f2c34-7b20-49cd-8b4b-8dc09ccb59db admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.407584 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.408228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.408498 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.408767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] PATH_INFO: `/v3/projects/17b8531a16c4491a8cd0737d7f11b126/users/7bd15fc30e3148ddaf13fc747bf1e3cd/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.410728 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17b8531a16c4491a8cd0737d7f11b126, user_id=7bd15fc30e3148ddaf13fc747bf1e3cd, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.413358 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.423551 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1386/2784] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:39:57 2022] PUT /identity/v3/projects/17b8531a16c4491a8cd0737d7f11b126/users/7bd15fc30e3148ddaf13fc747bf1e3cd/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:39:57.434171 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.434463 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.434740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.462941 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] MFA Rules not processed for user `7bd15fc30e3148ddaf13fc747bf1e3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:39:57.472597 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:57.538287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1399/2785] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:39:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:39:57.549867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.552170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:39:57.611627 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0018d-fbf6-4cae-bb84-451af452c63d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.616035 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.616649 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.616932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.617254 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.626774 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.628202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.637704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1387/2786] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:39:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:39:57.715070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:39:57.725822 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1ee6c-8e47-4e19-b54a-e8cd82171a06 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:39:57.729110 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830309a-3da7-4274-9fc1-31b1cf2f4ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:39:57.729704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830309a-3da7-4274-9fc1-31b1cf2f4ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:39:57.729978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830309a-3da7-4274-9fc1-31b1cf2f4ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:39:57.730247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830309a-3da7-4274-9fc1-31b1cf2f4ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:39:57.740464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:39:57.791699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830309a-3da7-4274-9fc1-31b1cf2f4ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:39:57.793147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830309a-3da7-4274-9fc1-31b1cf2f4ea9 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:39:57.802008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1400/2787] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:39:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:07.633488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:07.797964 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=22 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:36.528109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:36.589349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d547d-5f4e-4e3a-a934-f9073459c636 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:36.594138 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:36.594765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:36.595043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:36.595314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] PATH_INFO: `/v3/users/8e0c1ce44f8e475d95549802b4dca2a8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:36.596445 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0c1ce44f8e475d95549802b4dca2a8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:36.598401 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:36.644473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0c1ce44f8e475d95549802b4dca2a8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:36.652199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0c1ce44f8e475d95549802b4dca2a8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:36.653077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0c1ce44f8e475d95549802b4dca2a8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:36.653748 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0c1ce44f8e475d95549802b4dca2a8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:36.660746 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0c1ce44f8e475d95549802b4dca2a8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:36.678795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1388/2788] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:40:36 2022] DELETE /identity/v3/users/8e0c1ce44f8e475d95549802b4dca2a8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:36.683252 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6657c-de50-4542-9681-612029f4724d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:36.683766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6657c-de50-4542-9681-612029f4724d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:36.683766 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6657c-de50-4542-9681-612029f4724d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:36.702630 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b2b6657c-de50-4542-9681-612029f4724d None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:36.703839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2b6657c-de50-4542-9681-612029f4724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:36.762969 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1401/2789] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:36.772675 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:36.774170 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:36.830850 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9eb5e-d64a-4c3d-8303-4601c253fac3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:36.834981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:36.835588 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:36.835866 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:36.836129 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:36.845610 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:36.847042 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:36.857090 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1389/2790] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:37.133845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6657c-de50-4542-9681-612029f4724d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:37.135283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2b6657c-de50-4542-9681-612029f4724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:37.147483 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6657c-de50-4542-9681-612029f4724d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:37.152251 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:37.152809 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:37.153121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:37.153990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] PATH_INFO: `/v3/projects/df34e34dcb374c41bab3a990a2cf2ea7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:37.154682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df34e34dcb374c41bab3a990a2cf2ea7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:37.157364 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:37.213611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df34e34dcb374c41bab3a990a2cf2ea7', 'request_id': 'req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.219691 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df34e34dcb374c41bab3a990a2cf2ea7', 'request_id': 'req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.227727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df34e34dcb374c41bab3a990a2cf2ea7', 'request_id': 'req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.238228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] The token cache is being invalidate because project df34e34dcb374c41bab3a990a2cf2ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:40:37.238513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.239132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.240604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1402/2791] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:40:37 2022] DELETE /identity/v3/projects/df34e34dcb374c41bab3a990a2cf2ea7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:37.244836 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:37.254539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874f259-e5ae-4a22-85ee-06f6e1e43729 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:37.272589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:37.313186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:37.313819 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:37.314093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:37.314367 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] PATH_INFO: `/v3/users/7bd15fc30e3148ddaf13fc747bf1e3cd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:37.315539 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd15fc30e3148ddaf13fc747bf1e3cd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:37.319015 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:37.367204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd15fc30e3148ddaf13fc747bf1e3cd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.377029 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd15fc30e3148ddaf13fc747bf1e3cd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.377678 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd15fc30e3148ddaf13fc747bf1e3cd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.378414 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd15fc30e3148ddaf13fc747bf1e3cd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.388417 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84093fe8-2ba7-47a6-bd79-6efb64b691b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd15fc30e3148ddaf13fc747bf1e3cd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.407515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1390/2792] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:40:37 2022] DELETE /identity/v3/users/7bd15fc30e3148ddaf13fc747bf1e3cd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:37.696688 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:37.698225 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:37.716366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf9d4-d48d-4b67-a1eb-b17621886d14 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:37.779773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:37.780494 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:37.780773 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:37.781070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] PATH_INFO: `/v3/projects/17b8531a16c4491a8cd0737d7f11b126` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:37.782220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17b8531a16c4491a8cd0737d7f11b126)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:37.784510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:37.815367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:37.839283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17b8531a16c4491a8cd0737d7f11b126', 'request_id': 'req-a170ebab-c207-4169-9fd5-7c1e5d622ad5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.845754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17b8531a16c4491a8cd0737d7f11b126', 'request_id': 'req-a170ebab-c207-4169-9fd5-7c1e5d622ad5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.853981 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17b8531a16c4491a8cd0737d7f11b126', 'request_id': 'req-a170ebab-c207-4169-9fd5-7c1e5d622ad5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.866882 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] The token cache is being invalidate because project 17b8531a16c4491a8cd0737d7f11b126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:40:37.867174 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.867846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a170ebab-c207-4169-9fd5-7c1e5d622ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:37.869642 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1403/2793] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:40:37 2022] DELETE /identity/v3/projects/17b8531a16c4491a8cd0737d7f11b126 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:37.874463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:37.875018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:37.875018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:37.895004 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:37.896216 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:37.956697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1391/2794] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:37.964716 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:37.965019 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:37.965312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:37.986934 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:37.988077 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:37.996823 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1404/2795] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:40:38.001226 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.011640 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a01a-3057-451a-b305-2df0619b4bc9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.016234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.016817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.017124 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.017396 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.018031 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:40:38.018329 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.019914 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.031458 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1392/2796] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:40:38.035967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.045544 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fca85-02b3-44c6-bdc9-e0ba720a594e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.051137 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.051734 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.052054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.052271 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.053410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.055291 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.083621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1405/2797] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:40:38 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:38.087877 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.098220 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235ccec-1e14-4ae1-aac3-56b6a6da9a33 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.125316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:38.159000 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.159607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.159882 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.160151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.160778 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.162715 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.184385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1393/2798] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:40:38 2022] POST /identity/v3/users => generated 278 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:38.188532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.199648 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12d630-ae55-46cb-9c8a-83ae4262968e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.204457 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.205103 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.205407 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.205703 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.206352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.207998 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.218437 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1406/2799] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:38.222411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.232771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f06704-c3bb-4346-befb-ba84304b288c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.237554 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce21587-9602-461c-aad3-7a4239b0e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.238086 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce21587-9602-461c-aad3-7a4239b0e956 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.238359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce21587-9602-461c-aad3-7a4239b0e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.238628 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce21587-9602-461c-aad3-7a4239b0e956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.239272 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce21587-9602-461c-aad3-7a4239b0e956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.241409 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce21587-9602-461c-aad3-7a4239b0e956 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.253956 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1394/2800] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:38.259002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.269191 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eca1bf-dc32-4b73-92af-2e8a09fb638e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.273986 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.274623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.274896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.275166 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] PATH_INFO: `/v3/projects/932a3dfc88694464aa89c480d6a9fad7/users/12b89fd62d2e4057bf157db48f28b98b/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.289987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932a3dfc88694464aa89c480d6a9fad7, user_id=12b89fd62d2e4057bf157db48f28b98b, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.292615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.306740 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1407/2801] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:38 2022] PUT /identity/v3/projects/932a3dfc88694464aa89c480d6a9fad7/users/12b89fd62d2e4057bf157db48f28b98b/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:38.318333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.319182 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.319416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.353375 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] MFA Rules not processed for user `12b89fd62d2e4057bf157db48f28b98b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:38.354595 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:38.419437 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1395/2802] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:38.423982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.486380 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98943105-b6eb-4da9-8fa4-d18fa1afedd3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.491660 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.492192 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.492466 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.492737 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.493675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.495540 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.521697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1408/2803] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:40:38 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:40:38.528557 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.542508 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc60eb8-8191-4743-84b9-242dc1ac0ecc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.609186 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.609792 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.610065 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.610336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.610963 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.612863 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.635795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1396/2804] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:40:38 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:38.639783 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.650878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d964ad-9363-46fe-804a-a427e8d62434 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.655526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.656119 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.656397 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.656668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.657355 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.659053 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.671013 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1409/2805] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:38.675100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.686105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf99b38-a53d-4036-a4ce-31f0ca9bc4eb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.690851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.691497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.691773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.692043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] PATH_INFO: `/v3/projects/a5cb5cc8458148b9bcd6bda4feb609f8/users/fe451be2dae7431ba03f886369e45d2f/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.707673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cb5cc8458148b9bcd6bda4feb609f8, user_id=fe451be2dae7431ba03f886369e45d2f, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.710902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.723680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1397/2806] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:38 2022] PUT /identity/v3/projects/a5cb5cc8458148b9bcd6bda4feb609f8/users/fe451be2dae7431ba03f886369e45d2f/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:38.728381 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.738548 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aa453-d895-4dc4-870c-d9212e6c7463 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.759071 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:38.798887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1de4f-2b49-400c-a985-685f730d015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.799487 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1de4f-2b49-400c-a985-685f730d015d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.799762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1de4f-2b49-400c-a985-685f730d015d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.800026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1de4f-2b49-400c-a985-685f730d015d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.800663 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1de4f-2b49-400c-a985-685f730d015d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.802269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1de4f-2b49-400c-a985-685f730d015d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.812188 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1410/2807] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:38.816505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:38.821789 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:38.828288 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4737a3-60bd-4a08-997d-f8320f2f8c41 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:38.833500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:38.834646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.834920 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.835190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] PATH_INFO: `/v3/projects/a5cb5cc8458148b9bcd6bda4feb609f8/users/fe451be2dae7431ba03f886369e45d2f/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.837178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cb5cc8458148b9bcd6bda4feb609f8, user_id=fe451be2dae7431ba03f886369e45d2f, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:38.839815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:38.851167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1398/2808] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:38 2022] PUT /identity/v3/projects/a5cb5cc8458148b9bcd6bda4feb609f8/users/fe451be2dae7431ba03f886369e45d2f/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:38.915129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:38.915411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:38.915682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:38.944730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] MFA Rules not processed for user `fe451be2dae7431ba03f886369e45d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:38.945993 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:39.020632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1411/2809] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:39.032572 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:39.034424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:39.093515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9caa49-bace-4f28-a6bb-99b31b301480 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:39.097261 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:39.097856 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:39.098185 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:39.098390 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:39.107521 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:39.109073 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:39.118701 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1399/2810] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:39.195202 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:39.205086 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a26096-4d93-4677-bcb5-44c2681368d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:39.208631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d17cb-9732-48e6-aab8-4c7722d67263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:39.209920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d17cb-9732-48e6-aab8-4c7722d67263 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:39.210193 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d17cb-9732-48e6-aab8-4c7722d67263 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:39.210464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d17cb-9732-48e6-aab8-4c7722d67263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:39.266224 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d17cb-9732-48e6-aab8-4c7722d67263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:39.268620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d17cb-9732-48e6-aab8-4c7722d67263 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:39.278340 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1412/2811] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:49.114681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:49.274244 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:54.458721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:54.521522 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf68709-a563-4214-8583-de666085f54e service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:54.526036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:54.526662 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:54.526931 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:54.527197 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] PATH_INFO: `/v3/users/12b89fd62d2e4057bf157db48f28b98b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:54.528355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b89fd62d2e4057bf157db48f28b98b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:54.530296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:54.579841 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b89fd62d2e4057bf157db48f28b98b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:54.587752 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b89fd62d2e4057bf157db48f28b98b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:54.588481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b89fd62d2e4057bf157db48f28b98b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:54.589131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b89fd62d2e4057bf157db48f28b98b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:54.597110 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b89fd62d2e4057bf157db48f28b98b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:54.616228 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1400/2812] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:40:54 2022] DELETE /identity/v3/users/12b89fd62d2e4057bf157db48f28b98b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:54.620978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:54.621485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:54.621591 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:54.641012 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:54.642231 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:54.703371 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1413/2813] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:54.713714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:54.715181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:54.773105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4893-7fa0-4e91-9322-e006b714d48a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:54.776763 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:54.777386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:54.777660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:54.777932 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:54.788195 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:54.789688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:54.799410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1401/2814] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:55.073203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:55.074639 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:55.086513 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ee12e-bc3a-4ff4-8155-f136546ee671 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:55.091187 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:55.091813 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.092100 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.092367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] PATH_INFO: `/v3/projects/932a3dfc88694464aa89c480d6a9fad7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.093552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932a3dfc88694464aa89c480d6a9fad7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:55.095497 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:55.146198 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932a3dfc88694464aa89c480d6a9fad7', 'request_id': 'req-6909f8c5-397c-473e-a9c3-1b505cb00e65'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.152580 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932a3dfc88694464aa89c480d6a9fad7', 'request_id': 'req-6909f8c5-397c-473e-a9c3-1b505cb00e65'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.159917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932a3dfc88694464aa89c480d6a9fad7', 'request_id': 'req-6909f8c5-397c-473e-a9c3-1b505cb00e65'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.170702 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] The token cache is being invalidate because project 932a3dfc88694464aa89c480d6a9fad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:40:55.170992 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.171631 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.173125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1414/2815] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:40:55 2022] DELETE /identity/v3/projects/932a3dfc88694464aa89c480d6a9fad7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:55.177665 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:55.187196 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e523f-3846-4a13-8a29-6b2550b34ef2 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:55.205514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:55.246473 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:55.247092 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.247363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.247630 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] PATH_INFO: `/v3/users/fe451be2dae7431ba03f886369e45d2f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.248847 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe451be2dae7431ba03f886369e45d2f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:55.250766 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:55.296301 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe451be2dae7431ba03f886369e45d2f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.303949 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe451be2dae7431ba03f886369e45d2f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.304594 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe451be2dae7431ba03f886369e45d2f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.305246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe451be2dae7431ba03f886369e45d2f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.312120 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-511f0772-66ad-40d6-a32c-9b8762f38778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe451be2dae7431ba03f886369e45d2f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.330063 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1402/2816] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:40:55 2022] DELETE /identity/v3/users/fe451be2dae7431ba03f886369e45d2f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:55.611957 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:55.613518 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:55.629528 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f8c5-397c-473e-a9c3-1b505cb00e65 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:55.695008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:55.695714 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.696056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.696486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] PATH_INFO: `/v3/projects/a5cb5cc8458148b9bcd6bda4feb609f8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.698099 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5cb5cc8458148b9bcd6bda4feb609f8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:55.700358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:55.734628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:55.753836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5cb5cc8458148b9bcd6bda4feb609f8', 'request_id': 'req-246fe1ba-4311-4451-8e60-78af76dc82dd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.760390 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5cb5cc8458148b9bcd6bda4feb609f8', 'request_id': 'req-246fe1ba-4311-4451-8e60-78af76dc82dd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.768446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5cb5cc8458148b9bcd6bda4feb609f8', 'request_id': 'req-246fe1ba-4311-4451-8e60-78af76dc82dd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.779632 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] The token cache is being invalidate because project a5cb5cc8458148b9bcd6bda4feb609f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:40:55.779920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.780550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246fe1ba-4311-4451-8e60-78af76dc82dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:55.782081 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1415/2817] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:40:55 2022] DELETE /identity/v3/projects/a5cb5cc8458148b9bcd6bda4feb609f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:55.787706 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.788356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.788356 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.808061 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:55.809292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:55.868500 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1403/2818] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:40:55.876209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.876489 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.876758 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.898080 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:55.899285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:55.907222 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1416/2819] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:55.912107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:55.922057 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90e4c6-9bd8-43a3-af24-d7b5f80cc154 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:55.926581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:55.927211 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.927484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.927753 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.928393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:40:55.928694 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:55.930395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:55.941992 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1404/2820] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 12:40:55.946348 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:55.955764 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daa1dc-2ec3-4c91-8a76-8397074d9ba6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:55.961020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:55.961611 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:55.961887 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:55.962155 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:55.962790 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:55.964682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:55.993406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1417/2821] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:40:55 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:55.997469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.008163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139591-5838-48b9-b029-e5b4ad0a9aae admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.034798 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:56.067472 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.068600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.068871 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.069184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.069817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.071815 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.093286 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1405/2822] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:40:55 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:56.097593 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.108163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cfe4d-7362-4747-ac15-7447748c9a00 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.112880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.113503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.113776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.114041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.115525 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.116269 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.127474 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1418/2823] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:56.132107 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.142426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43f0e2-5113-404c-924b-3c928600b8c7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.147828 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e03569-97ec-47df-b26d-4589586dc408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.148428 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03569-97ec-47df-b26d-4589586dc408 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.148702 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03569-97ec-47df-b26d-4589586dc408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.148975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03569-97ec-47df-b26d-4589586dc408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.149646 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e03569-97ec-47df-b26d-4589586dc408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.151327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e03569-97ec-47df-b26d-4589586dc408 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.162589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1406/2824] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:56.166939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.179132 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d3ed-577a-4482-b055-bc96172d988e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.184723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.185557 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.185878 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.186313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] PATH_INFO: `/v3/projects/df6abdfdd33d4439a1d7f0829f853d04/users/e49c831772ec4e5298117145f1213835/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.215144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6abdfdd33d4439a1d7f0829f853d04, user_id=e49c831772ec4e5298117145f1213835, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.217865 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.232794 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1419/2825] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:56 2022] PUT /identity/v3/projects/df6abdfdd33d4439a1d7f0829f853d04/users/e49c831772ec4e5298117145f1213835/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:40:56.243009 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.243285 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.243607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.278234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] MFA Rules not processed for user `e49c831772ec4e5298117145f1213835`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:56.279477 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:56.344676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1407/2826] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:56.349300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.412352 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948ba1f-681c-461e-b21e-657b1f811d97 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.417059 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.417745 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.418026 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.418295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.418923 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.420824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.449979 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1420/2827] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:40:56 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:56.454137 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.464664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce194e-339b-4d30-9e08-451830362238 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.520765 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.521316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.521589 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.521855 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.522483 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.524365 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.544940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1408/2828] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:40:56 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:56.549111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.560047 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cefa4-4251-47d5-ac0e-ffec93af0ac9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.564895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.565510 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.565785 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.566056 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.566693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.568894 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.581069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1421/2829] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:56.585258 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.595331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44468115-d991-4660-85a5-84709159e5e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.600131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.600780 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.601093 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.601366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] PATH_INFO: `/v3/projects/14a7bda2ad98453fbae48d813bcff7df/users/6b470d0f91ce4ce2a03cedc01b3c75e5/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.616768 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7bda2ad98453fbae48d813bcff7df, user_id=6b470d0f91ce4ce2a03cedc01b3c75e5, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.619740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.633310 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1409/2830] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:56 2022] PUT /identity/v3/projects/14a7bda2ad98453fbae48d813bcff7df/users/6b470d0f91ce4ce2a03cedc01b3c75e5/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:56.637278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.647199 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f913839-15f2-4d8c-85c6-4b1aacf2bafb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.666781 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:56.706527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307765f-4983-4b0c-8a88-6ef3336a6931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.707126 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307765f-4983-4b0c-8a88-6ef3336a6931 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.707399 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307765f-4983-4b0c-8a88-6ef3336a6931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.707668 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307765f-4983-4b0c-8a88-6ef3336a6931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.708365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307765f-4983-4b0c-8a88-6ef3336a6931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.710054 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307765f-4983-4b0c-8a88-6ef3336a6931 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.720250 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1422/2831] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:56.725079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.730600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:56.737275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e86b9d-a3e3-4816-876c-02f1b002f14b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.743959 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.744655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.745139 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.745300 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] PATH_INFO: `/v3/projects/14a7bda2ad98453fbae48d813bcff7df/users/6b470d0f91ce4ce2a03cedc01b3c75e5/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.747343 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7bda2ad98453fbae48d813bcff7df, user_id=6b470d0f91ce4ce2a03cedc01b3c75e5, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.750067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.761950 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1410/2832] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:56 2022] PUT /identity/v3/projects/14a7bda2ad98453fbae48d813bcff7df/users/6b470d0f91ce4ce2a03cedc01b3c75e5/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:56.772389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.772578 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.772855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.803083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] MFA Rules not processed for user `6b470d0f91ce4ce2a03cedc01b3c75e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:56.804313 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:56.882092 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1423/2833] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 12:40:56.893981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:56.895790 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:56.956762 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd13ac-6403-4295-a5d4-2d4f68624c8f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:56.960541 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa7172-9acb-4843-bae8-6ae752a8de7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:56.961181 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa7172-9acb-4843-bae8-6ae752a8de7f service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:56.961457 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa7172-9acb-4843-bae8-6ae752a8de7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:56.961722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa7172-9acb-4843-bae8-6ae752a8de7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:56.971366 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa7172-9acb-4843-bae8-6ae752a8de7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:56.972803 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa7172-9acb-4843-bae8-6ae752a8de7f service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:56.983722 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1411/2834] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:57.014819 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:57.080638 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456befe2-5720-4e3b-ab90-5042eecf6114 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:57.085844 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:57.086471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:57.086750 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:57.087020 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] PATH_INFO: `/v3/users/e49c831772ec4e5298117145f1213835` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:57.088158 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49c831772ec4e5298117145f1213835)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:57.090111 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:57.135435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49c831772ec4e5298117145f1213835'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.136974 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:57.143503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49c831772ec4e5298117145f1213835'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.144288 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49c831772ec4e5298117145f1213835'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.144922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49c831772ec4e5298117145f1213835'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.152439 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49c831772ec4e5298117145f1213835'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.170319 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1424/2835] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:40:57 2022] DELETE /identity/v3/users/e49c831772ec4e5298117145f1213835 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:57.174955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:57.175237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:57.175510 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:57.194236 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:57.195463 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:57.254397 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1412/2836] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:57.264426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:57.265947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:57.323410 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db87325-e44d-42d7-8b42-d319686cd6a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:57.327070 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:57.327682 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:57.327947 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:57.328216 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:57.337829 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:57.339165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:57.348539 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1425/2837] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:57.615079 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:57.616497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:57.629704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ef5bf-18b3-4008-b39c-b9810c67ad56 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:57.635200 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:57.635860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:57.636136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:57.636489 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] PATH_INFO: `/v3/projects/df6abdfdd33d4439a1d7f0829f853d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:57.637549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6abdfdd33d4439a1d7f0829f853d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:57.639440 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:57.687316 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6abdfdd33d4439a1d7f0829f853d04', 'request_id': 'req-b9d58a13-e66b-4ba9-852a-02efb899443f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.693937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6abdfdd33d4439a1d7f0829f853d04', 'request_id': 'req-b9d58a13-e66b-4ba9-852a-02efb899443f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.701448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6abdfdd33d4439a1d7f0829f853d04', 'request_id': 'req-b9d58a13-e66b-4ba9-852a-02efb899443f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.712553 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] The token cache is being invalidate because project df6abdfdd33d4439a1d7f0829f853d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:40:57.712837 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.713512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.714984 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1413/2838] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:40:57 2022] DELETE /identity/v3/projects/df6abdfdd33d4439a1d7f0829f853d04 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:57.719300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:57.729046 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca8ce5-e12c-4cea-93d2-645586f06477 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:57.787164 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:57.787789 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:57.788066 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:57.788339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] PATH_INFO: `/v3/users/6b470d0f91ce4ce2a03cedc01b3c75e5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:57.789490 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b470d0f91ce4ce2a03cedc01b3c75e5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:57.791393 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:57.837283 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b470d0f91ce4ce2a03cedc01b3c75e5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.844615 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b470d0f91ce4ce2a03cedc01b3c75e5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.845253 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b470d0f91ce4ce2a03cedc01b3c75e5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.845846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b470d0f91ce4ce2a03cedc01b3c75e5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.853464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6173d862-90fc-4fce-b641-f02b284eee25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b470d0f91ce4ce2a03cedc01b3c75e5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:57.858846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:57.873604 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1426/2839] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:40:57 2022] DELETE /identity/v3/users/6b470d0f91ce4ce2a03cedc01b3c75e5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:58.150599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.153400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:58.167374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d58a13-e66b-4ba9-852a-02efb899443f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.181297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:58.231266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.231902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.232175 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.232493 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] PATH_INFO: `/v3/projects/14a7bda2ad98453fbae48d813bcff7df` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.233631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a7bda2ad98453fbae48d813bcff7df)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.235517 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.283658 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a7bda2ad98453fbae48d813bcff7df', 'request_id': 'req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:58.289681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a7bda2ad98453fbae48d813bcff7df', 'request_id': 'req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:58.296942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a7bda2ad98453fbae48d813bcff7df', 'request_id': 'req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:58.307454 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] The token cache is being invalidate because project 14a7bda2ad98453fbae48d813bcff7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:40:58.307738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:58.308359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be6f8b2-65cb-4c79-8b4a-1f419c7780e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:40:58.309883 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1414/2840] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:40:58 2022] DELETE /identity/v3/projects/14a7bda2ad98453fbae48d813bcff7df => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:58.314980 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.315285 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.315577 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.336266 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:58.337503 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:58.395477 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1427/2841] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:58.404721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.405020 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.405302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.424838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:58.426148 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:58.435660 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1415/2842] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:58.440157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.451353 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778a77-d839-4184-8ba9-0f6797aa3df6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.455982 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.456701 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.456976 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.457276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.457922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:40:58.458328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.460342 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.472646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1428/2843] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:40:58.476976 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.486157 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d340-ff18-4d47-a6f0-cf4127e49a10 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.490948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.491533 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.491806 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.492072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.492940 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.495155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.524361 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1416/2844] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:58.528723 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.539062 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6be6e-f361-4c79-93e1-a8337d27db6b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.595978 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.596581 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.596861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.597170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.597818 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.599803 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.622127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1429/2845] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:58.626167 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.637078 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf79ad-52a9-48f6-9bd3-a66c4432fc11 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.641669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.642259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.642538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.642812 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.643453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.645060 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.655038 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1417/2846] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:40:58.658890 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.669200 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893965ac-f0c5-4ed9-ac26-7e4a8125924c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.673908 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a1785-022d-41f2-8228-b2cef0f6e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.674511 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a1785-022d-41f2-8228-b2cef0f6e99d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.674786 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a1785-022d-41f2-8228-b2cef0f6e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.675052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a1785-022d-41f2-8228-b2cef0f6e99d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.675718 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a1785-022d-41f2-8228-b2cef0f6e99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.677851 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a1785-022d-41f2-8228-b2cef0f6e99d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.690301 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1430/2847] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:58.695113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.705568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad4800-cb86-4c51-b0fd-442e8e70e73f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.710199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.710833 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.711109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.711382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] PATH_INFO: `/v3/projects/4d802a1bd94b4ba78542ddde2e244d30/users/53d98cc90a104978a64d2878f1213b19/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.725969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d802a1bd94b4ba78542ddde2e244d30, user_id=53d98cc90a104978a64d2878f1213b19, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.728600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.742036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1418/2848] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:58 2022] PUT /identity/v3/projects/4d802a1bd94b4ba78542ddde2e244d30/users/53d98cc90a104978a64d2878f1213b19/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:58.753087 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.753573 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.753695 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.781543 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] MFA Rules not processed for user `53d98cc90a104978a64d2878f1213b19`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:58.782771 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:58.825274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:58.849550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1431/2849] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:58.854106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.900379 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:58.919202 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e045d81-f592-4896-ac4e-65dda0fd24ad admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:58.923795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:58.924381 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:58.924654 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:58.925474 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:58.925608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:58.927972 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:58.954633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1419/2850] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:58.958867 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:58.970744 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62164ee-a98e-4617-9b51-597e700de8d9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.028621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.029247 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.029527 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.029798 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.030421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.032328 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.053209 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1432/2851] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:40:58 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:40:59.057253 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:59.068514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48716fec-3f13-46a1-9ed4-697f0393abfd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.073479 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.074080 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.074362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.074635 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.075331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.076975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.088307 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1420/2852] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:59.092402 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:59.102409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddf1ed-19d1-4fce-b9a2-1e73d2dd1549 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.107045 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.107722 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.107971 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.108243 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] PATH_INFO: `/v3/projects/08f668642e83448dac3b8b8d9e3c68cd/users/6eace934db614bb187925dffb98d409e/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.124428 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f668642e83448dac3b8b8d9e3c68cd, user_id=6eace934db614bb187925dffb98d409e, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.127162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.141339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1433/2853] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:59 2022] PUT /identity/v3/projects/08f668642e83448dac3b8b8d9e3c68cd/users/6eace934db614bb187925dffb98d409e/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:59.145359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:59.156461 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428467c-afb0-4efd-9c36-fd21d9207781 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.213995 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b7c09-e756-43b8-a369-0033fc2e3fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.214592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7c09-e756-43b8-a369-0033fc2e3fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.214865 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7c09-e756-43b8-a369-0033fc2e3fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.215127 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7c09-e756-43b8-a369-0033fc2e3fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.215757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b7c09-e756-43b8-a369-0033fc2e3fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.217462 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b7c09-e756-43b8-a369-0033fc2e3fd7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.227199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1421/2854] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:40:59.231332 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:59.242320 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38905-3e24-4a78-a382-5b7a434f92f4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.247420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.248065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.248346 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.248645 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] PATH_INFO: `/v3/projects/08f668642e83448dac3b8b8d9e3c68cd/users/6eace934db614bb187925dffb98d409e/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.250697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f668642e83448dac3b8b8d9e3c68cd, user_id=6eace934db614bb187925dffb98d409e, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.253512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.264553 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1434/2855] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:40:59 2022] PUT /identity/v3/projects/08f668642e83448dac3b8b8d9e3c68cd/users/6eace934db614bb187925dffb98d409e/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:40:59.275838 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.276142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.276419 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.306600 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] MFA Rules not processed for user `6eace934db614bb187925dffb98d409e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:40:59.307825 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:59.384308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1422/2856] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:40:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:40:59.396542 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:59.398337 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:40:59.472008 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559b0b2-0638-4b55-975c-41bd4f0a5162 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.475897 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.476501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.476782 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.477085 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.486416 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.487840 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.498217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1435/2857] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:40:59.580476 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:40:59.590469 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9849b25-ce5e-4576-ab5c-4b90f0ee95aa None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:40:59.594119 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07e6fb-4a37-4439-bd25-3b70a1a0e1e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:40:59.594720 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07e6fb-4a37-4439-bd25-3b70a1a0e1e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:40:59.594994 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07e6fb-4a37-4439-bd25-3b70a1a0e1e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:40:59.595266 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07e6fb-4a37-4439-bd25-3b70a1a0e1e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:40:59.629298 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:40:59.654171 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07e6fb-4a37-4439-bd25-3b70a1a0e1e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:40:59.655273 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07e6fb-4a37-4439-bd25-3b70a1a0e1e1 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:40:59.666308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1423/2858] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:40:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:04.494157 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:04.536654 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:04.560836 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926dac-a50e-454d-b692-aa7d98f7bc23 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:04.566832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:04.567566 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:04.567845 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:04.568121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] PATH_INFO: `/v3/users/53d98cc90a104978a64d2878f1213b19` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:04.569289 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d98cc90a104978a64d2878f1213b19)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:04.571240 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:04.621814 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d98cc90a104978a64d2878f1213b19'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:04.629061 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d98cc90a104978a64d2878f1213b19'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:04.629681 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d98cc90a104978a64d2878f1213b19'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:04.630274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d98cc90a104978a64d2878f1213b19'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:04.637440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d98cc90a104978a64d2878f1213b19'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:04.654762 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1436/2859] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:41:04 2022] DELETE /identity/v3/users/53d98cc90a104978a64d2878f1213b19 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:41:04.659194 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:04.659672 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:04.659771 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:04.679648 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:04.680843 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:04.739650 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1424/2860] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:04.749970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:04.751376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:04.808824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1082-3554-4cfe-b945-e7d6ebbc4d0f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:04.812486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:04.813121 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:04.813400 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:04.813669 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:04.822776 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:04.824194 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:04.834129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1437/2861] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:05.097690 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:05.099222 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:05.112302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e05097-1ffb-411c-a593-2c2ba409b333 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:05.118081 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:05.119759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:05.120036 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:05.120302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] PATH_INFO: `/v3/projects/4d802a1bd94b4ba78542ddde2e244d30` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:05.121549 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d802a1bd94b4ba78542ddde2e244d30)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:05.123755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:05.171887 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d802a1bd94b4ba78542ddde2e244d30', 'request_id': 'req-96d20d0a-25b7-4157-92fe-bd1b94d38341'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.178052 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d802a1bd94b4ba78542ddde2e244d30', 'request_id': 'req-96d20d0a-25b7-4157-92fe-bd1b94d38341'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.184938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d802a1bd94b4ba78542ddde2e244d30', 'request_id': 'req-96d20d0a-25b7-4157-92fe-bd1b94d38341'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.196575 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] The token cache is being invalidate because project 4d802a1bd94b4ba78542ddde2e244d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:41:05.196857 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.197506 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.198969 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1425/2862] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:41:05 2022] DELETE /identity/v3/projects/4d802a1bd94b4ba78542ddde2e244d30 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:05.202880 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:05.212949 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d2d8a-e499-4720-a0de-4718fd774b13 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:05.270055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:05.271309 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:05.271582 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:05.271848 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] PATH_INFO: `/v3/users/6eace934db614bb187925dffb98d409e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:05.273234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eace934db614bb187925dffb98d409e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:05.275370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:05.276828 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:05.324427 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eace934db614bb187925dffb98d409e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.332217 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eace934db614bb187925dffb98d409e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.332920 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eace934db614bb187925dffb98d409e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.334142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eace934db614bb187925dffb98d409e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.341468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eace934db614bb187925dffb98d409e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.359022 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1438/2863] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:41:05 2022] DELETE /identity/v3/users/6eace934db614bb187925dffb98d409e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:05.541903 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:05.543535 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:05.614569 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d20d0a-25b7-4157-92fe-bd1b94d38341 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:05.627773 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:05.656932 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:05.658807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:05.680317 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:05.681099 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:05.681937 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:05.682304 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:05.684681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:05.697412 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:05.699376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54263be1-0eda-4f01-b237-cfee3381ac12 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:05.708831 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1426/2864] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:05.758417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12035e8f-e6dd-4c5a-8ec8-cb7cbbf429d9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:05.763228 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:05.763876 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:05.764149 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:05.764420 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] PATH_INFO: `/v3/projects/08f668642e83448dac3b8b8d9e3c68cd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:05.765602 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f668642e83448dac3b8b8d9e3c68cd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:05.767733 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:05.823378 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f668642e83448dac3b8b8d9e3c68cd', 'request_id': 'req-14f8fa39-20f8-4528-ae82-4879369be8af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.830326 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f668642e83448dac3b8b8d9e3c68cd', 'request_id': 'req-14f8fa39-20f8-4528-ae82-4879369be8af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.838720 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f668642e83448dac3b8b8d9e3c68cd', 'request_id': 'req-14f8fa39-20f8-4528-ae82-4879369be8af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.851406 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] The token cache is being invalidate because project 08f668642e83448dac3b8b8d9e3c68cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:41:05.851755 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.852624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-14f8fa39-20f8-4528-ae82-4879369be8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:05.854683 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1439/2865] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:41:05 2022] DELETE /identity/v3/projects/08f668642e83448dac3b8b8d9e3c68cd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:05.862205 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:05.862636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:05.862901 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:05.887704 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:05.889523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:05.952377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1427/2866] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:05.960896 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:05.961229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:05.961505 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:05.981870 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:05.983213 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:05.991646 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1440/2867] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:05.996337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.006655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7e44c-bc25-41e0-9289-021305be9473 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.011505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.012112 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.012386 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.012686 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.013362 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:41:06.013666 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.015265 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.027359 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1428/2868] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:41:06.032107 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.037374 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:06.044274 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d626fbb-871b-4841-b09a-4b87bf736325 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.050624 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.051278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.052156 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.052431 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.053134 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.055058 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.085024 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1441/2869] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:41:06 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:41:06.089337 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.101234 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df9d18-54a2-49af-9b6d-334a58744f04 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.159730 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.160334 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.160614 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.160954 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.162131 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.164184 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.187169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1429/2870] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:41:06 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:41:06.191526 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.202345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17c1d5-3788-4466-b5c6-c55cd15a2679 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.207050 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.207652 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.208460 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.208736 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.209414 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.210990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.225220 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1442/2871] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:41:06.226433 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.236681 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a626e9-968a-4525-8cd9-dd45d3367413 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.241822 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ca819-a27a-47d4-9887-bfd099ae0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.242444 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ca819-a27a-47d4-9887-bfd099ae0f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.242975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ca819-a27a-47d4-9887-bfd099ae0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.243268 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ca819-a27a-47d4-9887-bfd099ae0f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.243918 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ca819-a27a-47d4-9887-bfd099ae0f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.245581 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ca819-a27a-47d4-9887-bfd099ae0f32 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.257860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1430/2872] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:41:06.262005 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.273349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72818f1d-18fe-4d83-a461-9891ab3a7cd7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.278620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.279278 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.279558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.279834 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] PATH_INFO: `/v3/projects/3e3d0948d3bb43e58ef51a89860c6ff9/users/c11c4221e6684e5dab138234b4b5a6e0/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.297784 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3d0948d3bb43e58ef51a89860c6ff9, user_id=c11c4221e6684e5dab138234b4b5a6e0, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.300442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.315142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1443/2873] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:41:06 2022] PUT /identity/v3/projects/3e3d0948d3bb43e58ef51a89860c6ff9/users/c11c4221e6684e5dab138234b4b5a6e0/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:06.325332 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.325607 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.325896 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.356275 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] MFA Rules not processed for user `c11c4221e6684e5dab138234b4b5a6e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:06.357560 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:06.370174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:06.423336 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1431/2874] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:06.434450 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.436162 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:06.494444 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aabcc-7280-45f4-9793-e99ba88101d8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.498048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c97c-8750-4047-84f6-59127eec6153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840491b8d05489c8475e1125dbc0b5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.498640 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c97c-8750-4047-84f6-59127eec6153 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.498911 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c97c-8750-4047-84f6-59127eec6153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.499176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c97c-8750-4047-84f6-59127eec6153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.507807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c97c-8750-4047-84f6-59127eec6153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.509238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c97c-8750-4047-84f6-59127eec6153 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.517883 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1444/2875] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:06.540854 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.603996 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a7ed3-0469-4096-9446-a1d88f76d272 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.608495 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.609142 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.609416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.609682 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] PATH_INFO: `/v3/users/c11c4221e6684e5dab138234b4b5a6e0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.610834 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11c4221e6684e5dab138234b4b5a6e0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.612740 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.659123 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11c4221e6684e5dab138234b4b5a6e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:06.666276 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11c4221e6684e5dab138234b4b5a6e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:06.666981 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11c4221e6684e5dab138234b4b5a6e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:06.667601 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11c4221e6684e5dab138234b4b5a6e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:06.674633 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11c4221e6684e5dab138234b4b5a6e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:06.691759 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1432/2876] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:41:06 2022] DELETE /identity/v3/users/c11c4221e6684e5dab138234b4b5a6e0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:06.696886 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.697307 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.697486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.717089 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:06.718316 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:06.778065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1445/2877] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:06.788592 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:06.790018 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:06.824912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:06.853738 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c04e13-4a62-46a1-9e16-d14b0902864d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:06.857598 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9d551-31ba-4cd1-9220-dd729aa3d87f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:06.858302 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d551-31ba-4cd1-9220-dd729aa3d87f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:06.858719 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d551-31ba-4cd1-9220-dd729aa3d87f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:06.859163 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9d551-31ba-4cd1-9220-dd729aa3d87f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:06.869953 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9d551-31ba-4cd1-9220-dd729aa3d87f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:06.871393 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9d551-31ba-4cd1-9220-dd729aa3d87f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:06.882314 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1433/2878] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:07.172621 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.174120 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:07.188502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89766c3-9aca-4110-a2aa-b1c9f330a1ba None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.194558 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.195203 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.195485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.195912 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] PATH_INFO: `/v3/projects/3e3d0948d3bb43e58ef51a89860c6ff9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.197354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3d0948d3bb43e58ef51a89860c6ff9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.199298 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.246312 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:07.250475 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3d0948d3bb43e58ef51a89860c6ff9', 'request_id': 'req-5bd28bf5-1f22-4da1-af9d-18107af0350f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:07.258368 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3d0948d3bb43e58ef51a89860c6ff9', 'request_id': 'req-5bd28bf5-1f22-4da1-af9d-18107af0350f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:07.267201 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3d0948d3bb43e58ef51a89860c6ff9', 'request_id': 'req-5bd28bf5-1f22-4da1-af9d-18107af0350f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:07.278215 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] The token cache is being invalidate because project 3e3d0948d3bb43e58ef51a89860c6ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:41:07.278501 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:07.279125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5bd28bf5-1f22-4da1-af9d-18107af0350f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:07.280637 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1446/2879] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:41:07 2022] DELETE /identity/v3/projects/3e3d0948d3bb43e58ef51a89860c6ff9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:07.286488 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.286767 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.287085 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.307223 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:07.308385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:07.377852 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1434/2880] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:07.385675 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.385962 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.386238 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.407083 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:07.408323 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:07.417421 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1447/2881] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:07.421939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.433106 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdcc64-8551-4e67-8ccf-56f56f5301f2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.437405 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.437979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.438259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.438531 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.439212 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 12:41:07.439512 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.441133 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.453805 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1435/2882] 10.222.0.85 () {58 vars in 1153 bytes} [Wed Nov 2 12:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 12:41:07.458471 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.468383 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9c12-dc29-400f-a87a-2e831206af2e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.473796 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.474426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.474699 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.474970 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.475623 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.477555 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.506405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1448/2883] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:41:07 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:41:07.511116 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.521757 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3fafe-ae3c-40af-9dd2-3e7d2606c9d8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.580179 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.580816 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.581244 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.581513 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.582169 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.586410 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.609799 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1436/2884] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:41:07 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 12:41:07.614064 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.625839 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248a36-9f0d-4943-895e-8cc38ffbb267 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.631035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.631576 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.631858 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.632129 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.632780 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.634405 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.649295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1449/2885] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 12:41:07.650465 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.660786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310661f-318a-4720-9dba-8658a84c6a55 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.666113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3a144-97cc-4994-b96e-d223346c5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.666692 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3a144-97cc-4994-b96e-d223346c5ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.666965 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3a144-97cc-4994-b96e-d223346c5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.667277 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3a144-97cc-4994-b96e-d223346c5ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.667902 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3a144-97cc-4994-b96e-d223346c5ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.669661 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3a144-97cc-4994-b96e-d223346c5ce8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.680571 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1437/2886] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:41:07.684693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.695263 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6e238-d808-4c55-b6b3-7c115ebf1277 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.699973 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.700731 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.701065 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.701345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] PATH_INFO: `/v3/projects/fc45d759676e41178c93f56305f41958/users/88eb3c1d9e63402b8d045bc2f9e4d91a/roles/5ccb31a782a44a4aad62b2f6bba3d582` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.717142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc45d759676e41178c93f56305f41958, user_id=88eb3c1d9e63402b8d045bc2f9e4d91a, role_id=5ccb31a782a44a4aad62b2f6bba3d582)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.719753 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.733712 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1450/2887] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:41:07 2022] PUT /identity/v3/projects/fc45d759676e41178c93f56305f41958/users/88eb3c1d9e63402b8d045bc2f9e4d91a/roles/5ccb31a782a44a4aad62b2f6bba3d582 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:07.740318 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d8693-9627-407d-a08d-6bcd1a6d2d09 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.740783 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d8693-9627-407d-a08d-6bcd1a6d2d09 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.740881 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d8693-9627-407d-a08d-6bcd1a6d2d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.760754 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-838d8693-9627-407d-a08d-6bcd1a6d2d09 None None] MFA Rules not processed for user `16d94c9123ce4b32b47b3b412b38ab1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:07.762006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-838d8693-9627-407d-a08d-6bcd1a6d2d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:07.802281 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:07.824098 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1438/2888] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:07.834345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:07.836017 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:07.896704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e97c6-6ce2-458a-8d23-a0601ac7008d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:07.900357 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:07.900961 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:07.901281 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:07.901552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:07.910579 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:07.914170 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:07.925917 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1451/2889] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:12.045879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.046399 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.046487 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.087233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] MFA Rules not processed for user `88eb3c1d9e63402b8d045bc2f9e4d91a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:12.088480 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:12.162322 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1439/2890] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:12.166837 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:12.177098 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495c61-a325-42a2-a15e-bca84facb818 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:12.181922 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:12.182531 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.182812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.183084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.183895 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:12.185924 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:12.216411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1452/2891] 10.222.0.85 () {60 vars in 1153 bytes} [Wed Nov 2 12:41:12 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:41:12.220327 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:12.232155 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600160b1-3bc6-4ab8-97ad-aa516aac4157 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:12.295505 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:12.296113 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.296385 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.296676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.297376 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:12.299333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:12.321617 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1440/2892] 10.222.0.85 () {60 vars in 1144 bytes} [Wed Nov 2 12:41:12 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 12:41:12.325727 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:12.337048 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bed80-af78-4783-9e0b-7cd5d9e5f6ca admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:12.342127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:12.342812 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.343163 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.343532 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.344248 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:12.345903 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:12.358469 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1453/2893] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:41:12.362721 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:12.373978 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff49ea5-510a-4b37-b1e1-15d085152fb4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:12.378970 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:12.379671 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.380011 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.380233 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] PATH_INFO: `/v3/projects/b0a1a24ed40f4bbc88a0163fea3319e0/users/44de4bbc44bb4e07a3d16aacf32304bd/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.400249 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a1a24ed40f4bbc88a0163fea3319e0, user_id=44de4bbc44bb4e07a3d16aacf32304bd, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:12.410368 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:12.423612 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1441/2894] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:41:12 2022] PUT /identity/v3/projects/b0a1a24ed40f4bbc88a0163fea3319e0/users/44de4bbc44bb4e07a3d16aacf32304bd/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:12.427601 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:12.438791 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b5add-9e06-414e-9b05-2ebf5092c0ae admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:12.481127 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:12.511335 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2506-d801-4c02-9470-3414d5c65a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:12.511939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2506-d801-4c02-9470-3414d5c65a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.512214 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2506-d801-4c02-9470-3414d5c65a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.512484 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2506-d801-4c02-9470-3414d5c65a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.513165 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2506-d801-4c02-9470-3414d5c65a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:12.514832 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2506-d801-4c02-9470-3414d5c65a21 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:12.525999 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1454/2895] 10.222.0.85 () {58 vars in 1122 bytes} [Wed Nov 2 12:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 12:41:12.530912 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:12.543217 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e79da3-7149-42a3-8f00-f7935ad24d61 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:12.549355 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:12.550252 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:12.550523 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:12.550793 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] PATH_INFO: `/v3/projects/b0a1a24ed40f4bbc88a0163fea3319e0/users/44de4bbc44bb4e07a3d16aacf32304bd/roles/25bb4875811e4bacb09d3e72977a7b1f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:12.552781 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a1a24ed40f4bbc88a0163fea3319e0, user_id=44de4bbc44bb4e07a3d16aacf32304bd, role_id=25bb4875811e4bacb09d3e72977a7b1f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:12.555490 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:12.568240 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1442/2896] 10.222.0.85 () {60 vars in 1483 bytes} [Wed Nov 2 12:41:12 2022] PUT /identity/v3/projects/b0a1a24ed40f4bbc88a0163fea3319e0/users/44de4bbc44bb4e07a3d16aacf32304bd/roles/25bb4875811e4bacb09d3e72977a7b1f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:17.185676 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12624aa8-c10a-42ef-be35-5fd5355edefe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:17.186097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12624aa8-c10a-42ef-be35-5fd5355edefe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:17.186097 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12624aa8-c10a-42ef-be35-5fd5355edefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:17.223446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-12624aa8-c10a-42ef-be35-5fd5355edefe None None] MFA Rules not processed for user `44de4bbc44bb4e07a3d16aacf32304bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:17.224698 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-12624aa8-c10a-42ef-be35-5fd5355edefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:17.339515 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1455/2897] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:17.348077 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:17.349851 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:17.388134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:17.456636 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf894-cfcc-4b8f-9adf-6f7d39718c12 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:17.461481 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce6a8adf15e3498d95442b3ad0ef3864', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:17.462105 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:17.462382 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:17.462655 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:17.543862 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:17.545411 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:17.556321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1443/2898] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:17.708438 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254385ea-dd9e-4599-954b-bbd188c590b9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:17.708761 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254385ea-dd9e-4599-954b-bbd188c590b9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:17.709115 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254385ea-dd9e-4599-954b-bbd188c590b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:17.782088 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-254385ea-dd9e-4599-954b-bbd188c590b9 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:17.783415 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-254385ea-dd9e-4599-954b-bbd188c590b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:17.851950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1456/2899] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:17.908378 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:17.910159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:18.014259 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac23f9-a4e9-4b57-9b80-a063e5d7276f service glance] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:18.020442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:18.021048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:18.021308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:18.021673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:18.031784 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:18.033484 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:18.043515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1444/2900] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:41:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 12:41:26.899930 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:26.900556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:26.900556 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:26.919855 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] MFA Rules not processed for user `88eb3c1d9e63402b8d045bc2f9e4d91a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:26.921084 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:26.927951 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1457/2901] 10.222.0.85 () {58 vars in 958 bytes} [Wed Nov 2 12:41:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:27.192248 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:27.223296 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:27.248174 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d1895-fc52-4bdb-bd22-12f976ee4b23 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:27.251804 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:27.252398 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:27.252673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:27.252948 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:27.255333 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:27.264867 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:27.266297 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c8c9e-cd5e-4add-92ef-9c4d6b6610fd service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:27.274278 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1445/2902] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:27.618139 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:27.619552 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:27.643176 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:27.700919 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561423d-5ca3-4289-ac9f-9ca55b9f2384 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:27.706078 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:27.706711 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:27.706987 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:27.707261 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:27.709069 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:27.721546 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:27.723587 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:27.735279 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1458/2903] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:29.068151 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:29.068689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:29.068689 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:29.089442 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:41:29.090688 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:29.098608 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1446/2904] 10.222.0.85 () {58 vars in 1009 bytes} [Wed Nov 2 12:41:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:41:29.119142 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:29.137354 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd3b31-f425-4427-8cec-a8a571174107 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:29.142846 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:29.143462 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:29.143738 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:29.143989 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:29.154509 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:29.156144 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:29.165551 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1459/2905] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:41:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:41:38.938237 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:38.940404 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:39.087290 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436140f7-2930-4fdb-a6ce-038f052279c5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:39.093190 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:39.093817 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:39.094094 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:39.094363 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:39.096204 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:39.110345 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:39.112879 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:39.131320 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1447/2906] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:39.161925 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=31 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:42.552610 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:42.586424 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c40763-abd9-443d-b97e-fc116bf7169e service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:42.614369 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:42.615524 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:42.616043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:42.616596 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:42.619370 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:42.634464 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:42.636304 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:42.651972 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1460/2907] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:46.173546 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:46.178006 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:46.210102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db62f5b-34f1-4c68-a19f-42f3b74d4a8d service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:46.230747 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c173404c311b43e29193004dc41480b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:46.231860 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:46.232264 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:46.232619 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:46.235424 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:46.253964 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:46.255735 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:46.279669 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1448/2908] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:41:52.409535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:52.411857 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:52.488027 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984f014-f52f-4204-bc31-884ce7ac552f service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:52.493125 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:52.493754 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:52.494028 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:52.494295 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] PATH_INFO: `/v3/users/ff4810b8d4ef4c80a47b3208ce5439d5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:52.495411 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4810b8d4ef4c80a47b3208ce5439d5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:52.497560 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:52.549644 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4810b8d4ef4c80a47b3208ce5439d5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.557300 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4810b8d4ef4c80a47b3208ce5439d5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.557910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4810b8d4ef4c80a47b3208ce5439d5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.558512 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4810b8d4ef4c80a47b3208ce5439d5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.565516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4810b8d4ef4c80a47b3208ce5439d5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.584535 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1461/2909] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:41:52 2022] DELETE /identity/v3/users/ff4810b8d4ef4c80a47b3208ce5439d5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:52.777999 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:52.789942 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ad655-7536-4572-b08b-0ffe5e2dedf0 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:52.859408 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:52.860199 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:52.860599 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:52.860955 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] PATH_INFO: `/v3/projects/2b3b8f23a66e4540a0f65330c6d6496e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:52.862711 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3b8f23a66e4540a0f65330c6d6496e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:52.865515 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:52.919146 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:52.928152 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3b8f23a66e4540a0f65330c6d6496e', 'request_id': 'req-42bb8b61-f247-4d6b-9707-a8acb74b2b20'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.934497 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3b8f23a66e4540a0f65330c6d6496e', 'request_id': 'req-42bb8b61-f247-4d6b-9707-a8acb74b2b20'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.945303 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3b8f23a66e4540a0f65330c6d6496e', 'request_id': 'req-42bb8b61-f247-4d6b-9707-a8acb74b2b20'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.958938 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] The token cache is being invalidate because project 2b3b8f23a66e4540a0f65330c6d6496e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:41:52.959232 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.959892 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:52.961453 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1449/2910] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:41:52 2022] DELETE /identity/v3/projects/2b3b8f23a66e4540a0f65330c6d6496e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:52.965550 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:52.967052 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:52.981260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b594f8-dc56-4eda-9e06-71b760ff19b7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:53.047516 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:53.048075 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:53.048349 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:53.048620 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] PATH_INFO: `/v3/users/c1175476deaf4f14967accc136a0b823` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:53.050367 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1175476deaf4f14967accc136a0b823)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:53.052697 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:53.062060 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:53.112743 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1175476deaf4f14967accc136a0b823'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.122520 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1175476deaf4f14967accc136a0b823'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.123486 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1175476deaf4f14967accc136a0b823'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.124276 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1175476deaf4f14967accc136a0b823'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.133234 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1175476deaf4f14967accc136a0b823'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.155968 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1462/2911] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:41:52 2022] DELETE /identity/v3/users/c1175476deaf4f14967accc136a0b823 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:53.409374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:53.411141 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:53.426606 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb8b61-f247-4d6b-9707-a8acb74b2b20 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:53.489697 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:53.490349 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:53.490625 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:53.490895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] PATH_INFO: `/v3/projects/daf9d2d6e1ee4ec09ab4a8ef642f93ec` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:53.492330 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf9d2d6e1ee4ec09ab4a8ef642f93ec)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:53.494293 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:53.560043 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf9d2d6e1ee4ec09ab4a8ef642f93ec', 'request_id': 'req-8b086ab7-616a-41fc-9864-d73d276b181c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.570102 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf9d2d6e1ee4ec09ab4a8ef642f93ec', 'request_id': 'req-8b086ab7-616a-41fc-9864-d73d276b181c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.578400 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf9d2d6e1ee4ec09ab4a8ef642f93ec', 'request_id': 'req-8b086ab7-616a-41fc-9864-d73d276b181c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.589894 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] The token cache is being invalidate because project daf9d2d6e1ee4ec09ab4a8ef642f93ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:41:53.590178 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.590840 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:41:53.592321 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1450/2912] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:41:53 2022] DELETE /identity/v3/projects/daf9d2d6e1ee4ec09ab4a8ef642f93ec => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:41:58.481950 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:41:58.483446 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:58.541426 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ee9ec-6554-4474-bf45-52109420c2dc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:41:58.589541 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e102dacae430417aa150ddd53cf34f31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:41:58.590147 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:41:58.590423 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:41:58.590693 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:41:58.592442 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:41:58.644485 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:41:58.658035 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:41:58.659959 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f04d54-56a4-4644-a055-c2c6d14f9406 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:41:58.669633 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1463/2913] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:41:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:42:03.586455 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:42:08.665409 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:43:06.734395 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:06.736067 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:06.827786 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b086ab7-616a-41fc-9864-d73d276b181c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:06.834108 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:06.835048 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:06.835416 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:06.835922 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:06.837975 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:06.852928 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:06.854874 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:06.866426 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1451/2914] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:43:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:43:12.148861 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:12.149449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:12.149449 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:12.169268 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] MFA Rules not processed for user `b5445ced373148aeb5741e1e11e897cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 12:43:12.170473 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:12.228067 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1464/2915] 10.222.0.85 () {58 vars in 967 bytes} [Wed Nov 2 12:43:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 12:43:12.269676 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:12.271109 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:12.341072 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd147-4c70-4a22-8437-a66eeac46d5f service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:12.346514 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef33e7dcb56a481aa051c09d45bfc83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:12.347134 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:12.347434 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:12.347708 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:12.356925 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:12.359418 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:12.368348 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1452/2916] 10.222.0.85 () {60 vars in 1384 bytes} [Wed Nov 2 12:43:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 12:43:15.810730 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:15.839815 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817d259-3ce2-47f4-b271-83722c7c2bd2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:15.845672 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:15.846358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:15.846704 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:15.847063 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:15.848910 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:15.868287 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:15.871365 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:15.885990 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1465/2917] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:43:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:43:19.052209 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:19.065377 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd5406-1614-40ea-9977-e686f3ccb2d7 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:19.070452 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:19.071136 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:19.071407 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:19.071673 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:19.073664 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:19.086255 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:19.087849 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:19.104028 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1453/2918] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:43:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:43:19.113435 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:19.116358 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:19.192169 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92aab6-d930-447b-9dbb-26e284d130bc service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:19.197763 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:19.198417 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:19.198696 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:19.198967 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] PATH_INFO: `/v3/users/88eb3c1d9e63402b8d045bc2f9e4d91a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:19.200094 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88eb3c1d9e63402b8d045bc2f9e4d91a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:19.202055 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:19.253366 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88eb3c1d9e63402b8d045bc2f9e4d91a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.262041 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88eb3c1d9e63402b8d045bc2f9e4d91a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.262939 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88eb3c1d9e63402b8d045bc2f9e4d91a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.263260 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88eb3c1d9e63402b8d045bc2f9e4d91a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.272160 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88eb3c1d9e63402b8d045bc2f9e4d91a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.291389 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1466/2919] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:43:19 2022] DELETE /identity/v3/users/88eb3c1d9e63402b8d045bc2f9e4d91a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:43:19.473568 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:19.487448 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31bf32-8a00-49ec-accd-9fcc2ae48294 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:19.545755 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:19.546911 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:19.547159 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:19.547538 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] PATH_INFO: `/v3/projects/fc45d759676e41178c93f56305f41958` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:19.548680 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc45d759676e41178c93f56305f41958)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:19.550631 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:19.558724 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:43:19.601308 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc45d759676e41178c93f56305f41958', 'request_id': 'req-cd8e0c0d-6b66-4511-9431-51f49287b31e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.607519 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc45d759676e41178c93f56305f41958', 'request_id': 'req-cd8e0c0d-6b66-4511-9431-51f49287b31e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.616464 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc45d759676e41178c93f56305f41958', 'request_id': 'req-cd8e0c0d-6b66-4511-9431-51f49287b31e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.627907 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] The token cache is being invalidate because project fc45d759676e41178c93f56305f41958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:43:19.628229 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.628895 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.630421 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1454/2920] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:43:19 2022] DELETE /identity/v3/projects/fc45d759676e41178c93f56305f41958 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:43:19.634824 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:19.636229 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:19.649232 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759a129-6b97-4107-af9a-801271217bf9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:19.657032 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:43:19.710002 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:19.710628 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:19.710988 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:19.711255 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] PATH_INFO: `/v3/users/44de4bbc44bb4e07a3d16aacf32304bd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:19.712372 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=44de4bbc44bb4e07a3d16aacf32304bd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:19.714347 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:19.763468 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44de4bbc44bb4e07a3d16aacf32304bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.770713 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44de4bbc44bb4e07a3d16aacf32304bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.771345 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44de4bbc44bb4e07a3d16aacf32304bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.772018 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44de4bbc44bb4e07a3d16aacf32304bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.779440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44de4bbc44bb4e07a3d16aacf32304bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:19.798502 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1467/2921] 10.222.0.85 () {58 vars in 1224 bytes} [Wed Nov 2 12:43:19 2022] DELETE /identity/v3/users/44de4bbc44bb4e07a3d16aacf32304bd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 12:43:19.989567 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:43:19.991331 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:43:20.005246 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e0c0d-6b66-4511-9431-51f49287b31e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:43:20.066507 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16d94c9123ce4b32b47b3b412b38ab1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbb8bdae345348eca7be88e01d7954f8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:43:20.067374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:43:20.067374 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:43:20.067642 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] PATH_INFO: `/v3/projects/b0a1a24ed40f4bbc88a0163fea3319e0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:43:20.068795 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a1a24ed40f4bbc88a0163fea3319e0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:43:20.070714 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:43:20.117121 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a1a24ed40f4bbc88a0163fea3319e0', 'request_id': 'req-6782dc04-4cfd-424a-8ebc-c359ea080629'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:20.123100 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a1a24ed40f4bbc88a0163fea3319e0', 'request_id': 'req-6782dc04-4cfd-424a-8ebc-c359ea080629'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:20.131241 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a1a24ed40f4bbc88a0163fea3319e0', 'request_id': 'req-6782dc04-4cfd-424a-8ebc-c359ea080629'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:20.141935 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] The token cache is being invalidate because project b0a1a24ed40f4bbc88a0163fea3319e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 12:43:20.142292 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:20.142979 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6782dc04-4cfd-424a-8ebc-c359ea080629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 12:43:20.144456 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1455/2922] 10.222.0.85 () {58 vars in 1233 bytes} [Wed Nov 2 12:43:19 2022] DELETE /identity/v3/projects/b0a1a24ed40f4bbc88a0163fea3319e0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 12:43:29.794339 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:43:30.138939 dv-cinder-863273-1-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 12:48:12.181769 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 12:48:12.183212 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:48:12.326043 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be8075-b5f5-4b03-8431-0607db5d8dea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 12:48:12.386647 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1de535a7ce1e475abdb02308cb3326e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c95872158724e60be029b048d666cf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 12:48:12.387258 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 12:48:12.387537 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 12:48:12.387808 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 12:48:12.389649 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 12:48:12.401807 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 12:48:12.403376 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a9cdb-7242-4cd7-ab1a-b9d83add8f2b service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 12:48:12.413440 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1468/2923] 10.222.0.85 () {60 vars in 1365 bytes} [Wed Nov 2 12:48:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 12:48:22.409049 dv-cinder-863273-1-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}